Transforming Your Data with Python
This post is part 3/3 in the enDAQ in Python Series
Before you receive your enDAQ sensor, your device is calibrated by providing a predefined vibration to the device that the sensor’s raw data will be adjusted to match. However, given the environments that accelerometers go through—harsh shocks, vibration, high temperatures, etc—the accelerometer unit’s sensitivity can significantly change overtime. To ensure you confidence about your data’s accuracy, we recommend sending your unit back for recalibration once a year.
In this recalibration process, we generate new coefficients in the same manner as was done initially. However, implementing these new calibration polynomials on your device does not adjust your data in your previously already recorded files—so in the event that you notice changes in your data that warrant recalibration, you may look to replace the old calibration polynomials on the recent files with the newer coefficients from recalibration. This can be done via the enDAQ Lab software manually, or for greater speed or to build a project on top of, can also be done programmatically in a Python environment using our open-source endaq.ide library.
In this post, I'll cover:
- Video
- Recap to Get Started
- Tables of Channels & Basic Info
- Accessing the Transform
- Changing the Transform
Recap to Get Started
To start organizing information from your IDE files, we’ll use our endaq.ide library again to grab our file and collect some sample channels to work with. To understand the ide and channels objects, check out our Introduction to IDEs post.
There are two types of calibration polynomials we can expect when working with our sensors—Univariate and Bivariate transforms. Our Bivariate transforms include coefficients that compensate for temperature that affects the accelerometer, such as those that are Piezo-Resistive.
New transformations should be applied to the subchannels individually rather than to the overall channel, as the parent channel’s transformation typically represents the unit conversion equation.
Accessing the Transform
The transform return, in this case a Univariate object, shows you how the calibration equation is applied to the raw data. The coefficients themselves can be grabbed (and changed as you’ll see further down) using its coefficients attribute.
Here you can see how the Bivariate equation considers another variable y for temperature data. When receiving calibration gain and offset coefficients for Bivariate channels, your gain and offset are your second and fourth coefficients above.
Changing the Transform
For a point of comparison before we actually change the transform, we should view the current values of the data--this can be done using our to_pandas() DataFrame return. To see more of how to use this feature and some of our other DataFrames options, check out the previous part of this series: A Guide to Organized Data with Pandas DataFrames
Changing the transform is done by reassigning the coefficients. The coefficients will need to be in a tuple of the same size as the original set of coefficients, which can then be set in place using the <Subchannel>.updateTransforms() method.
Now when viewing the values in the subchannel's DataFrame, the numbers reflect the change in the transform. It is important to note that transforms applied to an IDE through Python like this are never saved to the device or the IDE file! Reopening this IDE will restore the original calibration polynomials.
enDAQ in Python Series
This post is part of the enDAQ in Python Series. Feel free to check out my other related posts;
- Intro to IDE's, Channels, and Subchannels - Part 1/3
- Pandas DataFrames - Part 2/3
- Applying Transforms to Your Data - Part 3/3
- Recording Data Hands-Free
