Enkiworks Kalibrator

[Enki]

Currently working on writing a log parser that will consume huge random AP logs and shit out a decent MAF calibration. I'm having to learn C# in the process, so it is rather painful.

This is where you (the reader) come into play; continued harassment will result in me eventually finishing this thing, which subsequently will mean no more manual MAF calibrations will be required on your part.

Here's a screenshot of the pretty near finished UI; I will likely replace the "enki" picture with more options down the road, but a simple setup will suffice for now.



For those that give a shit, I had gotten pretty far with the script only version, but found it to be infeasible to collate and average all the data from even a small log file using only scripting. Thus, the switch to C#, which I am pretty positive will ultimately prove successful.

Feel free to use this thread to suggest features, report bugs (when I start doing betas), ask questions, or post power ballads about well hung llamas.


REQUESTED FEATURES:
Easier initial scaling for big MAF housings
Allow for all closed or open loop operation without app shitting itself in endless loops


NOW RUNNING TRIALS!

I will be taking a COUPLE of samples from people (that can/will do a manual MAFCAL themselves) whom post in this thread the following information (within CODE tags; keep the tabs in there or my lazy ass will skip you):

1. Current MAF calibration
2. Current WOT AFR targets
3. Accelerator pedal open loop transition
4. Load open loop transition
5. Rev limit
6. The log file
7. Your manually created MAF calibration (for me to compare)

Note that the log file needs at least the following columns within it or it will not work at all (extra data doesn't matter, but fewer extra columns results in better data resolution and thus, higher accuracy):
Accel. Pedal Pos. (%)
Actual AFR (AFR)
Boost (PSI)
Calculated Load (Load)
Long Term FT (%)
MAF Voltage (V)
RPM (RPM)
Short Term FT (%)
Wastegate Duty (%)

Order of these columns within the log file does not matter. Also, please have everything in one post so I can go back to it later for further testing.

[Lex]

Excellent! This will be a nice little app.

[chuckms6]

ive been comtemplating trying to code something like this since i started visual basic class, we just havent learned how to process large amounts of data like that yet lol

[Enki]

Well in order to do it accurately (which is kind of a given) you need to go by .01 volts on the MAF scale; this means interpolating a FUCKLOAD of data if your dataset is small, or estimating (which is what generally is done on manual MAFCals).

What I was about to do in script is create a 500 row by 4 column table for the end result data...This didn't include the datasets where I take each individual .01 volt increment worth of data from the log and average it, remove outliers, classify as OL/CL, etc before dumping it into the table proper.

Basically, I gave up on the scripted portion after running through a simulated processing path in my head and figuring out that my computer would shit ram all over the place and bring a single thread to its knees trying to get through even a smallish log.

At least this way, I can use linq or at worst load the log into an (embedded) SQL backend for query rather than having to pre-sort before manipulation.

TLDR:
Old code would result in this:


Also, I am a huge nerd.

[86azms3]

Sounds homo James

[MattJackson86]

Let me know if you need any help on this. Probably don't have the time to code it, but questions or if you need a 2nd set of eyes to look it over. I was looking to work on something like this (and other auto tune stuff) with me standback software I wrote.

[Enki]

Originally Posted by 86azms3 Sounds homo James

Eat a bowl of dicks Corey

Originally Posted by MattJackson86 Let me know if you need any help on this. Probably don't have the time to code it, but questions or if you need a 2nd set of eyes to look it over. I was looking to work on something like this (and other auto tune stuff) with me standback software I wrote.

I'm learning as I go. I can read the languages just fine, it's getting my feet wet getting started on writing that's a pain in the ass. Already almost done with regex validation for the MAF table and AFR inputs:

MAF Values ([0-9]+\.[0-9][0-9][\t]){128,}?[0-9]+[\.][0-9][0-9]
WOT AFR Values ([0-9][0-9]\.[0-9][0-9][0-9][\t]){13,}?[0-9][0-9][\.][0-9][0-9][0-9]

[MattJackson86]

This helps me with RegEx: Regex Tester Online

[Enki]

Already checked my regex; tis good.

[leon78]

We're not Worthy!!!! - YouTube

We're not Worthy!!!! - YouTube

[06Speed6]

This is going to be so nice, the one thing that I am starting to really hate is doing maf cals, its never perfect and it takes hours to even get it close.

[BigRedSpecial]

I had no idea something like this was even possible... Now what excuse will I have to randomly go out and burn half a tank of gas?

[MS3-a(sore)ASS-rex]

I think you should change the picture with "ENKI" in it to o e of T-Pain...he is the king of autotune after all.

In all seriousness though, this will be fucking amazing if you can get it to work.

[Enki]

Originally Posted by 06Speed6 This is going to be so nice, the one thing that I am starting to really hate is doing maf cals, its never perfect and it takes hours to even get it close.

I personally don't think it's *THAT* bad, but I'd rather drive around running errands for up to an hour and drop the whole fat log into an app like this and have it shit out a pretty good MAFCAL.

Originally Posted by BigRedSpecial I had no idea something like this was even possible... Now what excuse will I have to randomly go out and burn half a tank of gas?

Anything is possible if you put your mind to it. Also, half a tank will probably be too big of a log (unless you're on full e85, in which case the fuel gauge will drop quickly). What kind of space does the AP have on it for logs, anyways?

Originally Posted by MS3-a(sore)ASS-rex I think you should change the picture with "ENKI" in it to o e of T-Pain...he is the king of autotune after all. In all seriousness though, this will be fucking amazing if you can get it to work.

That picture is already gone; I found something useful to put there after all. I really hate wasted space in my user interfaces, so I'll look for anything to stuff into open space or sometimes even completely re-design the UI to make it more compact and user friendly.



As for it working, it will; at some point anyways. My biggest hurdle right now is knowledge, but I'm already up to 200 lines of UI validation and content conformance (making sure people are passing the right data to the app). I probably need to make the regex a bit more strict, and plan ahead for larger MAF housings though.

As a side note, I already plan on adding wastegate integration in the future (making hitting those boost targets super easy); that will probably only be for WOT though, as running the entire table will be a pain in the ass.

[BigRedSpecial]

Originally Posted by Enki Anything is possible if you put your mind to it. Also, half a tank will probably be too big of a log (unless you're on full e85, in which case the fuel gauge will drop quickly). What kind of space does the AP have on it for logs, anyways?

lol... Not what I meant at all... I mean, if I can use logs from daily driving to calibrate my maf, what excuse will I use to leave the GF home at night to go out and just drive around for two hours?

[Enki]

You could still tell her you're going out to get logs for tuning/calibrating. That part still hasn't changed.

[Lex]

Be careful with delays if you're using STFTs or even AFRs since during throttle transitions these can really vary.

I would look for steady state throttle, then look at LTFTs, and then STFTs as well. I would average STFTs (just like the car does) in broken up regions of the MAF curve that is more granular than what the current LTFTs do.

[Enki]

Well the plan is to average all but outlier data (stuff that exceeds a certain threshold of difference from the meat of the data) and use 100% of the LTFT but only 50% of STFT data (for now; thanks phate for this idea!). I figure, if you want it more accurate, you can simply pull another log and run it back through again.

The whole averaging thing means the larger and more diverse the raw log file is, the more accurate this will become; doing the short little logs isn't going to cut it as there won't be enough intelligence in the code to discern situational changes like throttle lift or roll on, etc.

BTW, still making progress; currently loading logs into datatables for parsing!

[rfinkle2]

Wow Enki. To learn C++ just to bring this out is a feat all in itself. Thanks very much for this.

Arguably the most important part of the tune is being addressed with much less chance of error.

This is something that I will definitely use.

Thanks very much. Lots of effort.

[phate]

Subbed. The CL/OL transition range is always the trickiest part. You might want to incorporate some limit as to how much change between adjacent cells can occur. [I believe the derivative of the maf curve is almost linear, so you might just be able to assume this and limit it with a single value.]

IF someone has a leak, you get very bumpy maf curves if you tune around it. That should NOT be done, and the curve should always stay pretty dang smooth. If the calculated correction in one area falls outside of the norm, red flags and whistles should go off, because their setup needs inspected.

I foresee this tool being used (unknowingly) by folks to tune around things that should not be tuned around.

Edit: start praying to the gods of internal combustion for automatically calibrating a maf curve while spraying meth.

[Jake_MS3]

Subbed for future reference. Thanks for all the hard work and time we all know you're spending!

[2k4_8]

Wow,bthis will sure be nice! Another humble thanks!

[Enki]

Originally Posted by phate Subbed. The CL/OL transition range is always the trickiest part. You might want to incorporate some limit as to how much change between adjacent cells can occur. [I believe the derivative of the maf curve is almost linear, so you might just be able to assume this and limit it with a single value.] IF someone has a leak, you get very bumpy maf curves if you tune around it. That should NOT be done, and the curve should always stay pretty dang smooth. If the calculated correction in one area falls outside of the norm, red flags and whistles should go off, because their setup needs inspected. I foresee this tool being used (unknowingly) by folks to tune around things that should not be tuned around. Edit: start praying to the gods of internal combustion for automatically calibrating a maf curve while spraying meth.

I'm making this more for the pro-tuners (whom I assume will drive the feature-set forward), and the noobies (whom will be reading/accepting a big fat list of caveats before allowed to even obtain this).

For now, final determination of acceptability will be on the end user; run at your own risk sort of thing. Ultimately, it shouldn't be too hard to leave calibration A as default and drop the apps output into B, and flip back and forth to determine if you want to use it or not.

Your input has been taken into advisement and will be considered (if I can figure out a nice, safe way to implement it) for inclusion in a later version.

Thanks for the input Phate!

[Monotonous ONE]

i want to suck your cock

[silvapain]

Originally Posted by phate Subbed. The CL/OL transition range is always the trickiest part. You might want to incorporate some limit as to how much change between adjacent cells can occur. [I believe the derivative of the maf curve is almost linear, so you might just be able to assume this and limit it with a single value.] IF someone has a leak, you get very bumpy maf curves if you tune around it. That should NOT be done, and the curve should always stay pretty dang smooth. If the calculated correction in one area falls outside of the norm, red flags and whistles should go off, because their setup needs inspected. I foresee this tool being used (unknowingly) by folks to tune around things that should not be tuned around. Edit: start praying to the gods of internal combustion for automatically calibrating a maf curve while spraying meth.

The 2nd derivative of the MAF curve is linear. I checked it out a while ago using numerical analysis methods.


I tried out a mathematically ideal MAF curve once. Didn't work too well.


Tapadatass

[Enki]

Originally Posted by Monotonous ONE i want to suck your cock

Well shit, I guess I HAVE to finish it now (preferably while you finish me! )

[mrdouble99]

Totally awesome

[Enki]

Originally Posted by silvapain The 2nd derivative of the MAF curve is linear. I checked it out a while ago using numerical analysis methods.

Care to elaborate? Last math class I had in high school the final project was to balance a checkbook.

[phate]

Originally Posted by silvapain The 2nd derivative of the MAF curve is linear. I checked it out a while ago using numerical analysis methods. I tried out a mathematically ideal MAF curve once. Didn't work too well. Tapadatass

Ah, yeah. We talked about that at one point.

[silvapain]

Originally Posted by Enki Care to elaborate? Last math class I had in high school the final project was to balance a checkbook.

I'm way too lazy to type it all out on my phone, but a derivative of a function with respect to an input is the change in that function's output WRT the input (I.e. the slope of the line for a given x). A numerical solution involves taking known discrete points (x1, y1), (x2, y2) and determining the changes in output (y2-y1) based on the change in input (x2-x1). That is the 1st derivative. repeat the process again for the 2nd derivative.


Tapadatass

[Lex]

It will probably be easiest if the user does a MAF cal log that has very gentle throttle inputs and that extends into WOT to cover the whole range.

Have them log desired AFR as well as actual AFR along with the LTFT, STFT information.

[TexasSpeed3]

damn this is making me want to get back into programming. but shit so many variables in this one. your a brave soul taking on learning a new language as well as this program by itself. cant wait to see how this works

[SoAgg159]

Gonna be badass, shit, az will have something to be proud of.

Can you please change the ENKI on the front to a unicorn shooting jizz outta his dick with a rainbow behind him?

[Enki]

Originally Posted by Lex It will probably be easiest if the user does a MAF cal log that has very gentle throttle inputs and that extends into WOT to cover the whole range. Have them log desired AFR as well as actual AFR along with the LTFT, STFT information.

Gen 1 doesn't have commanded AFR parameter, sadly.
So far the required logged values are accel pedal position, afr, boost, calc load, both trims, maf voltage, rpm and wastegate. I thought about doing mass airflow, but it didn't seem to line up with the voltage sometimes.

As a side note, hit my first big hurdle and am currently working through it.

[06Speed6]

I wonder if someone could write a script that would check for improper curves that are out of a certain range. There has gotta be a way because that script would throw a flag for meth systems, aux fueling, intake leaks, and maybe some other stuff im not thinking of.

The math is way over my head but I know we have some math gurus around here that can make graph formula that fits.

[Enki]

Originally Posted by 06Speed6 I wonder if someone could write a script that would check for improper curves that are out of a certain range. There has gotta be a way because that script would throw a flag for meth systems, aux fueling, intake leaks, and maybe some other stuff im not thinking of. The math is way over my head but I know we have some math gurus around here that can make graph formula that fits.

Across the maf scale, there is a maximum jump from voltage to voltage. It should be possible to average these jumps out, and if higher than a certain percentage (a base/ots mafcal can be used for this), a flag can be thrown. Ultimately, it will be obvious as the curve will suddenly jump by a good amount when viewed in ATR.

[mrmonk7663]

Hellz yah. I've been waiting 2 months for an update on this...I'm glad you are making progress on it and will continue to wait for the beta release. The incorporation of larger maf housings...3", 3.25, 3.5, and 4" would be great!!!

Good Job Enki.

[ms6forthewin]

wow! this would be great!!

[JasonMzdSpd3]

Sick! This should be one kick ass app when complete.

Sent from Epic 4G via tap & touch 2.

[SRTie4k]

If you need any help with your methodology or have any questions in general, let me know. I'm just finishing up a class in C++, and although C# takes a departure from C++ in certain areas, they are relatively similar.