Introduction to IDEs for Data Analysis in Python

Introduction to IDE files and Python

This post is part 1/3 in the enDAQ in Python Series

If you're looking to get started with IDE files and Python, I've put this video and detailed post together to help guide you through the process and start your journey working with your enDAQ shock, vibration and environmental recordings.

Your enDAQ is structured such that the sensors on your device will take measurement recordings and report the data straight to “channels” that are further separated into subchannels for the separate event types gathered from an individual sensor. The event types that separate into subchannels of a sensor can look like the X, Y, and Z axis events from an accelerometer, or pressure, temperature, humidity from a control pad sensor.

This means that to get to applying your own analytics to the data from your recordings, you’ll need to find a means to access the digital channels that hold the data—which is now made easy through our endaq.ide open-source library.

However, to get to channel data, the class hierarchy must be followed first by accessing a Dataset object that represents your entire IDE file, storing all your channels, and your channels will have to be adjusted to separate out sensor information from the data itself.

The video walks you through all the steps, functions, and methods needed to be able to work your way to your data while also walking you through the other information that can be grabbed along the way that may help with how you organize your projects.

In this post, I'll cover:

• Video
• Installing endaq.ide
• Getting an IDE & Its Channels
• Basic Info from the IDE (Dataset Object)
• Accessing a Channel
• Accessing a Subchannel
• Accessing More from Channels & Subchannels - Sessions!

Install endaq.ide

To be able to work with your finished IDE recording files, you’ll need to start with our endaq.ide Python library that can be installed via pip.

The hierarchy starts with Dataset objects that represent an entire IDE file. These then contain our channels and subchannels with our data that we can grab only once we have our Dataset object.

Getting an IDE & Its Channels

To grab the IDE file as a Dataset object, we use endaq.ide's get_doc() function and provide the filename. If the file isn't in the working directory with your Python script, you'll need to provide a full path. To collect Channel objects, we can use get_channels().

The Channels list return can be narrowed to providing a specific measurement via the measurement_type parameter. The endaq.ide.measurement module offers a selection of MeasurementType objects that can be specified (and combined), though you can also specify this as a string argument. If the subchannels parameter is not specified, only the relevant subchannels in the Dataset will be returned.

Basic Info from the IDE (Dataset Object)

In our framework, Sensor objects symbolize the physical components present on our enDAQs. However, they do not store any recorded signal data themselves. Instead, the Channel objects serve as the hub where these individual sensors “report” their data to.

Basic information regarding the file location and name can be grabbed as well.

Accessing a Channel

Accessing a Subchannel

Channels and subchannels are derived from a common parent class, allowing you to utilize the same attributes for both, with the exception of the “.subchannels” attribute specific to channels.

You’ll notice we still haven’t gotten to anything related to the data itself! This is because to do so, the EventArray object must be grabbed out of the channels/subchannels. EventArrays that represent your channel or subchannel allow you to access the data itself and more data-specific information as well. The session return of a channel and a subchannel are both EventArray instances; thereby each of the following methods can be applied with either type of EventArray.

Accessing More from Channels & Subchannels - Sessions!

Using arrayRange() or arraySlice() on a channel EventArray will return a 2D array that contains a time series as well as, for each subchannel, an array of each event per time value in the time series. For a subchannel EventArray, the returns will include only the events coming from that single subchannel along with the time series. To view other formats to view and use your data, check out our post on DataFrames in enDAQ. 

An event is a measurement at a single time. A subchannel’s event is always one measurement per time, a channel’s event holds measurements from all subchannels at a single time. The index of an event before or near a specific time can be grabbed using the following, where the time provided is in microseconds. 

enDAQ in Python Series

This post is part of the enDAQ in Python Series. Feel free to check out my other related posts;

• Intro to IDE's, Channels, and Subchannels - Part 1/3
• Pandas DataFrames - Part 2/3
• Applying Transforms to Your Data - Part 3/3
• Recording Data Hands-Free