Tuesday, May 18, 2010

MySQL table is full error for InnoDB

I was playing with importing data from csv into a table that is only in memory with InnoDB and it came up with an error:
"table is full"
There was 107,000 records added and it stopped before finishing the import.

-- here is the sql code that I used to make the table
SELECT 'ITEM_IMPORT' AS "Drop Table";
DROP TABLE IF EXISTS item_import;
-- ------------------------------------------------------------------
-- Create ITEM_IMPORT table.
-- ------------------------------------------------------------------
SELECT 'ITEM_IMPORT' AS "Create Table";
CREATE TABLE item_import
( item_barcode CHAR(14) NOT NULL
, item_type INT UNSIGNED NOT NULL
, item_title CHAR(60) NOT NULL
, item_subtitle CHAR(60)
, item_rating CHAR(8) NOT NULL
, item_release_date DATE NOT NULL
, screen_type INT UNSIGNED DEFAULT NULL
) ENGINE=memory;

-- loads the csv into the temporary table
LOAD DATA LOCAL INFILE 'C:/Documents and settings/mike/Desktop/datawarehousing/import.csv'
INTO TABLE item_import
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
ESCAPED BY '\\'
LINES TERMINATED BY '\r\n';

to fix the limit on the amount of rows in the InnoDB table change or add the following line to the my.ini if you are using windows:
max_heap_table_size = 600M
I tested this and it enabled me to add 4 million rows in the table. Since it is a table in memory it removed the tables data as soon as you exit mysql.

No comments: