Skip to main content

Vehicle File Import

Introduction​

This will import vehicle file to Cloud SQL for PostgreSQL

Fields​

Column PositionDescriptionDatatypeValidationMax lengthMandatoryExample
AVehicle Reg NumberStringA-Z and 0-9 & All Special Char10YABC123
BVehicle IdNumber0-95Y1
CVINStringA-Z and 0-917YABC2D4GHY47524999

Importing Vehicle Files​

This process imports vehicle registration files from Modulus into database. Modulus provides separate vehicle registration files for Peugeot and Citroen brands.

Step 1: Preparation​

Before importing new vehicle data, you need to clear existing data and prepare the files:

1.1 Clear Existing Data​

  • Truncate the existing data in the modulus_vehicle table

1.2 Prepare Import Files​

  • You will receive two separate files from Modulus:
    • One for Peugeot vehicles
    • One for Citroen vehicles
  • Important: Import both files one after the other (not simultaneously)

1.3 Upload Files to Storage​

  • Upload each file to the Google Cloud Storage bucket: fnp-imports/modulus
  • Required filename: vehicle_reg.csv (rename each file to this exact name before uploading)
  • Process one brand at a time

Step 2: File Validation (Optional)​

  1. Navigate to Cloud Scheduler
  2. Locate and run the Modulus_vehicle_file_validation job
    • Note: Job name may vary slightly in different environments
  3. Wait for validation to complete successfully before proceeding

Step 3: Import Process​

  1. In the same Cloud Scheduler console
  2. Locate and run the Modulus_vehicle_file_import job
    • Note: Job name may vary slightly in different environments
  3. Monitor the import job for completion

Step 4: Repeat for Second Brand​

After successfully importing the first brand's file:

  1. Upload the second brand's file (rename to vehicle_reg.csv)
  2. Repeat Step for the second file
  3. Ensure both Peugeot and Citroen data are imported

Troubleshooting Memory Issues​

⚠️ Memory Error Handling:

If you encounter a memory error like Memory limit of 512 MiB exceeded with 515 MiB used, follow these steps:

  1. Increase the memory allocation for the import service (above 512MB)
  2. Re-run the import job
  3. Revert the memory back to default (512MB) after successful import

Testing​

  1. Data will be inserted to Cloud SQL for PostgreSQL under fpn database in table modulus_vehicle.
Status: Accepted
Category: Protected
Authored By: Gladson on Jan 5, 2024
Revisions