Importing CSV's

[greenamex2]

Hi

What is the cheapest way of doing a one off set of CSV imports in to i2?

I have a a bunch of data logging from my old Pi system that I have "exported" (via a powershell script...don't get me started!) to a CSV file. Would like to read them in i2 because it is a lot better than Pi toolbox.

Would rather they were in Standard so I could compare against my new data logging, if possible, but that seems to involve an expensive Pro license for someone like me (one man band just doing my own car, not a tuning shop or team).

Thanks in advance.

[adrian]

Firstly, when you say ".csv" file what exactly is that? Has your script created MoTeC formatted .csv files from the Pi files? You can get the MoTeC format by exporting any log file you have in i2 to .csv. You an open that to see how the csv needs to be formatted to get the import to work. As a side note, don't use Excel to edit csv files, it changes the format of the files and you won't be able to import them into i2.

If you don't want to convert your files to the MoTeC csv format then we have a Pi ASCII importer which you can use directly without having to convert to csv first.

In terms of getting the files imported, there are a number of MoTeC Dealers who have import licences who you could pay to convert the files for you or we offer 30day licences which you could use to convert the files yourself.

The resulting .ld files will be i2 Pro files, this means anyone can open them in i2 Standard or i2 Pro there is no additional licence required.

The way the Pro/Std files works is that if you have a Pro file (like the one from the import) you can open it in i2 Pro or i2 Std without the need of any other licences. If you have a Std file then you can open it in i2 Std but if you want to open it in i2 Pro then you need a licence.

[greenamex2]

Thanks Adrian.

Didn't realise standard can open pro...and of course all the sample logs are pro so it obviously works!

Unfortunately the lite version of Toolbox can't create ascii or csv files, hence the powershell script. I am copying the example import file formats so it will be compatible. Fortunately the Pi pdb file format is pretty simple, fairly standard index based, so the script wasn't that complicated...come across far worse in my time!

Guess I just need to find a willing dealer to take my money!

Better get that script finished and start enjoying i2.

Thanks for the help.

[greenamex2]

Hi

Anyway of picking up a lap beacon, distance or time when importing from a Motec CSV?

Or do I really have to manually enter lap times from Toolbox via a conversion spreadsheet?

Thanks in advance

[David Ferguson]

You can bring in multiple laps, I made some notes about converting AIM files to Motec using CSV export/import that I sent to one of my customers. This might be helpful to you:

- AiM’s “MoTeC CSV format” isn’t exactly compatible with the MoTeC CSV import (Beacon Markers need to be separated by spaces instead of comma-space). Each exported file needs to be edited so that I2 imports more than one lap. Single Lap export and import will work fine. I suggest you start by exporting a single lap, and importing that — I think that works without any editing — if you want to do multiple laps you need to edit the line that starts “Beacon Markers” to look like:

"Beacon Markers”, “290.488 628.015 739.839" (or whatever your values are)"

- You can’t edit a CSV file with Excel then write it out as a CSV, the file is changed (missing quotes around data fields) in ways that prevent I2 from importing it.

- Other spreadsheets (OpenOffice Calc, and Google Sheets) also can’t be used (various issues).

- You can edit the files with Windows Notepad app — but this does take some time (a few minutes each) to load and save the files.

- The exported data files are really big (20 - 100 MB each). One reason is that I exported all the channels (even the math channels), and since you had shock data I exported at a 100 hz sample rate. Normally, only those channels would be at the higher sample rate, and other channels would be lower — but when converted to CSV format, there is one row with all the channels at whatever sampling rate you specify. Sample Rate has the biggest effect on file size, and 20hz will be fine for comparing speed traces and most driver controls.

The good news — I2 will import the data quite quickly, and it can do an entire folder (and subfolders) without having to do individual files (ie, it’s fast and easy when given good data).

— The Data / Time of the logged data was the time of the import, not the date/time specified in the file (I reported that as a bug to MoTeC in March 2018).
— The event name/session doesn’t appear anywhere.
— Both of these can be fixed easiest within the I2 Details — but I have to provide the correct information for each file (I can get that from a RS2).

[greenamex2]

Thanks David. Tried it but it doesn't seems to be picking up the laps...any idea what I have done wrong?

This is the first few lines of the CSV file. I had to add the beacon line as it wasn't in the example I cribbed -

"Driver","Denis Bassom",,,"Engine ID","4A-GE"
"Device","PI_CL"
"Comment","Qual",,,"Session","1"
"Log Date","03/06/2012",,,"Origin Time","0.000","s"
"Log Time","0:00:01 AM",,,"Start Time","0.000","s"
"Sample Rate","10","Hz",,"End Time","529.6","s"
"Duration","529.6","s",,"Start Distance","0","m"
"Range","entire outing",,,"End Distance","9873.42","m"
"Beacon Markers","156.295 69.125 69.226 66.130 167.754"


"Time","Gear","Engine RPM","Ground Speed","G Force Lat","Eng Oil Pres","Engine Temp","Eng Oil Temp","Fuel Pres","Bat Volts ADL","Lap Distance","Throttle Pos","Damper Pos FL","Damper Pos FR","Damper Pos RL","Damper Pos RR","Damper Velocity FL","Damper Velocity FR","Damper Velocity RL","Damper Velocity RR","Brake Pres Front","Brake Pres Rear","Brake Bias Setting","Exhaust Temp Bank 1","Exhaust Temp Bank 2","Steered Angle","Fuel Tank Level","Avg Fuel Econ","Fuel Used","Laps Remaining","Beacon","Lap Time","Corr Dist"
"s","none","rpm","km/h","G","kPa","C","C","kPa","V","m","%","mm","mm","mm","mm","mm/s","mm/s","mm/s","mm/s","kPa","kPa","%","C","C","deg","l","m/l","l","none","none","s","m"


"0","1","1112","0","0.17","102","67.9","24.9","357","10.73","0","77.9","0","0","0","0","-3600","-3600","-4800","-6100","0","0","0","0","0","-46.6","0","0","0","0","0","0","0"
"0.1","1","1579","0","0.17","102","67.9","24.9","357","10.73","0","88.4","0","0","0","0","-1200","4800","1200","3600","0","0","0","0","0","-46.6","0","0","0","0","0","0","0"
"0.2","1","2071","0","0.02","189","67.9","24.9","365","11.07","0","87.9","0","0","0","0","-6100","-4800","-4800","-6100","0","0","0","0","0","-46.6","0","0","0","0","0","0","0"
"0.3","1","2575","0","0.02","189","67.9","24.9","365","11.07","0","62.9","0","0","0","0","-2400","-2400","-4800","-3600","0","0","0","0","0","-46.6","0","0","0","0","0","0","0"
"0.4","1","2806","0","0","326","67.9","24.9","358","11.29","0","30.4","0","0","0","0","-2400","-3600","-3600","-3600","0","0","0","0","0","-46.6","0","0","0","0","0","0","0"
"0.5","1","2442","0","0","326","67.9","24.9","358","11.29","0","0.4","0","0","0","0","4800","3600","3600","2400","0","0","0","0","0","-46.6","0","0","0","0","0","0","0"



Oh, and if you want a decent line editor try "Notepad++". Lots of great features. For example, I currently have over 20 CSV's open at once and can do text replace across all of them simultaneously.

[David Ferguson]

I think you need to have one overall time field, the Beacon list is the time at which each beacon occurs. Notice my example was increasing values. Yours appear to be lap times.

[greenamex2]

Thanks, got it. Combination of the beacon numbers...and having a channel called Beacon!

Time to rework the Powershell script a bit!

[greenamex2]

All done, thanks for the help.

Bye bye PI Toolbox, prefer I2 SO MUCH MORE!