Versions Compared

Old Version 18

changes.mady.by.user Scott Koranda

Saved on

compared with

New Version Current

changes.mady.by.user Benn Oshrin

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Note

Be careful with file ownerships here. You will need to run this command as the web server user or as root, depending on whether or not the web server user can read the input file. If running as root, make sure any files generated in the cache directory (wherever app/tmp points to) owned by root are subsequently deleted. 

...

BulkLoad can process ~500k records on a single vCPU with 2GB of RAM and a local database in about 10 minutes. Larger datasets may require somewhat more memory, but additional vCPUs are unlikely to help much. Make sure the database server has sufficient disk space available. (A few GB should be sufficient, depending on the size of the dataset.) Communications to a database server over a network (vs local to the same server) may result in slower run times.

See also: Registry Installation - PHP (Memory Considerations)

Steps

  1. Make sure access to the web interface is unavailable, if anyone else might try to use it.
  2. Run the BulkLoad command, specifying the target CO ID to load the records to. Depending on the number of records to load, the process will take a few minutes to complete.
    1. ./Console/cake bulk_load coid infile.json
  3. If the web interface is unusually slow after the load is completed, try clearing caches and running the cake database command to rebuild the indexes again.

...