using interrupts vs. polling of ADS1299

hitheshaum

I have a 32bit version that was working fine until yesterday. I wanted to change the chipkit code to get the data from ADS1299 via interrupts instead of polling it.  I reprogrammed my board many times to develop this. I have got a code that I think that works but suddenly the ADS_DRDY pin wouldn't go low at all. I couldn't figure out where the problem was and so I reprogrammed the board with the original code but still it doesn't work. It connects to the Processing GUI without any problem but wont give any data. I am disappointed only a little because I lost my 500$ board but mainly because I couldn't test my code. Anybody who resolved similar problem please comment how you resolved it. 

EDIT: Sorry my bad. I was using the wrong library settings in the tools.

wjcroft

OK, so you are saying you resolved this yourself?

hitheshaum

Yes. I resolved it. Also I managed to program a working code using interrupts. I am still testing it and i will post it on my github once I fully tested it. If someone need the code early, message me.

wjcroft

That sounds great, thanks. Can you tell us your motivation for using the interrupt scheme? Will that allow you to drive other IO or compute intensive algorithms on the board, relaxing some of the timing requirements?

Regards,

William

hitheshaum

Yes. Exactly. It will allow me to write time consuming algorithms to run smoothly on the board. I wanted to run FFT and if the data keeps coming at 250Hz, I will have less than 4ms to FFT the data, which is very less. If I use interrupts and if I wanted to do 256 point FFT, then I will have 256 times more time available to run my FFT. 

wjcroft

Nice. Keep us posted in this thread on your progress.

biomurph

@hitheshaum that's great!

We haven't gone the route of using interrupts because of all the other things that have had to be done.

Please join the community, if you haven't already, and you will be able to post your project on our community page

http://docs.openbci.com/tutorials/10-Community_Instructions

http://openbci.com/community/

JakeStew

I'm not understanding the logic of using interrupts, can you explain more on this?

Seems to me that it's not a standard interrupts vs. polling argument since you're not polling the status of anything, just requesting the new data when ready for it.  Seems like a wash to me at best.

How are you dealing with interrupts if/when generated before the processor is ready for new data?

hitheshaum

there is a pin ADS_DRDY attached to ADS chip that will go low whenever the data is ready. We just have to program an interrupt on falling edge of that particular pin.

JakeStew

So I assume that the occurrence of this is controlled by the sample rate set in the ADS1299 registers?

I'm still a bit confused about the implementation.

How is there any advantage or difference between polling at 256 Hz or getting interrupts at 256 Hz?

I'd love to hear a different take on how I would design the system since there are so many different ways to implement it.  I love to do interrupt based programs as much as possible, but it just seems to me that it might be easier to poll for data when the uC is ready rather than be interrupted upon the first nS that the data is available.

I'd imagine that either way would face opposing problems.  With interrupts the uC might not be ready for new data when interrupted, with polling the ADS1299 might not have the new data ready yet.  Seems to me the specific details of the implementation would make the difference on which strategy works better.

hitheshaum

@JakeStew Yes. There might be a problem when microcontroller is not ready for new data and it is interrupted. But this can be solved by simply storing the data and using it when ever the microcontroller is ready. This will need a bit more memory than with polling. But with polling, you can't do it at a certain rate. Porcessor simply waits and so you will be wasting a lot of precious time (or) you might induce delays in the data collection and so it won't be realtime anymore. Using interrupts is similar to using a parallel thread. You will get synchronization issues but will make the processor work at its full capacity.

wjcroft

Yes, my impression is that the interrupt routine would just store each successive sample into a buffer and then exit immediately. Then the background compute intensive task (such as FFT or other DSP), could just go about it's work with no concern about locking out samples during long calculations. Of course the background and interrupt layers would have to be carefully interlocked so they are not both fiddling with the buffer pointers at the same time.

So what this does is allow more decoupling between the 250/sec samples and whatever background tasks are consuming that stream. The background task still 'polls' the buffer pointer, but at it's convenience, rather than needing to sprinkle hardware poll checks throughout whatever large amount of background code there may be.

With the default firmware, this is not so much an issue. But if you are trying to run your own compute-intensive sketch at the same time -- this can buy some decoupling.

JakeStew

Whatever works!  All depends who you want clocking the cycle.  No matter which way you do things you only have 1/256th of a sec to do your calculations or you'll quickly overrun your ring buffer, so I don't see any real advantage either way.

I probably would clock off the ADS1299 interrupt line since that would save a timer on the uC.

hitheshaum

The interrupt based code is now available on my github (https://github.com/Hitheshaum/OBCI_Interrupt). It does exactly the same thing as the previous code does but now it uses interrupts instead of polling so that now you have the entire loop() at your disposal. If you update your firmware with this code you should not see any difference in terms of functionality. I did some basic testing. Feel free to test more contact me if you have any suggestions. 

biomurph

Hey, that's great hitheshaum!

If you are not using the SD card, you can also remove that code and library inclusion to gain more code headroom.

hitheshaum

@biomurph Well, my goal in using interrupts is to make it possible to use OpenBCI board standalone. So I would like to have some onboard data storage. Actually it only makes more sense now to use the SD card. 

@JakeStew I don't understand what you meant by ring buffer. Please note that I am using an 'external interrupt' and not the 'timer interrupt'

JakeStew

Your ISR should be reading and storing the data in a ring buffer then exiting.  The main loop will use this data as needed.

If your main loop does not use the data before the buffer is filled the data will be overwritten by new incoming data.

A ring (circular) buffer is just an array and two pointers.  The first pointer points at the last bye that was written.  The other points at the last byte that was read.  It's usually used when you want two threads independently reading and writing data, like in your interrupt case.

I'm not sure of any other normal way to do it.  You can use a buffer, but that doesn't change the fact that you still have to use/store the data at least as fast as it comes in or you'll have a buffer overrun or you'll have to drop data.  When a ring buffer over runs you can drop the oldest bytes by continuing or the newest data by stopping.  With most types of FIFO you'd usually drop the newest data because you'd stop adding in new data.

Do you have more details on what you want the main loop to actually be doing?  You could probably make the whole SD card into a large ring buffer.  If you're doing realtime display or something like that you can then easily see how far behind the data you are by just subtracting the read pointer from the write pointer.  Then you could jump ahead if you fall too far behind.

hitheshaum

@JakeStew Thanks for explaining me what a ring buffer is. I guess you are over-worried. Lets just say we need to perform a 256 point FFT. We will have to create two arrays int data[256], data_copy[256]; The ISR updates the data and when it completed updating data[255], all the data is copied to data_copy and the update pointer points back to data[0]. Now you can perform FFT on data_copy. 

The data_copy gets overwritten only when the next 256 data updates are run i.e 256 x time_between_each_update. If we take the data rate is 256Hz*, we will have a full 'one second' to do our FFT. 

But if you are polling the data, you will have to finish the FFT in 1/256 seconds or else you will lose the realtime updates. 

*Note: The OpenBCI data rate is 250Hz. The value 256 was taken just for simplicity. 

JakeStew

I still don't understand what you're trying to say.  There's no difference between the two methods other than potentially saving a timer on the uC and avoiding a few clock cycle penalty if you try to get the data early.

You're only saving a couple clock cycles per sample, at best.  With the context switching and ISR overhead you may even be losing a few. 

wjcroft

Jake, imagine you have a large compute intensive library routine you are calling. Such as a signal processing library that you have no control over and do not wish to modify (by placing polling checks for new samples coming in.) Say that some of these calls could take several hundred milliseconds. (New samples are arriving every 4 ms.)

The interrupt scheme will allow samples to not be lost while the heavy compute tasks are running. When those library calls complete, control goes back to the main loop where the interim samples are now handled or buffered.

JakeStew

Yes, trying to randomly or constantly poll with other stuff going on would not work well at all.

Microchip has pretty good RTOS support though if we really don't want to do basic housekeeping ourselves.  Just using an interrupt for the data acquisition doesn't really solve the problem of doing long blocking calls when we have several time sensitive independent tasks running at once.  It's a start though!

roman3017

@hitheshaum thank you very much for the code. you have opened up new possibilities for us. your work is much appreciated.

ntlanh

Hi @hitheshaum,

Thank you for your sharing of the code. As my understanding, doing interrupts helps reduce the time of getting the new data on the board.

Would it help increase the speed of writing the data to the SD card when increasing the sampling rate using your modified code?

wjcroft

The idea of the interrupt routine is to allow data acquisition to go on in parallel with other processes. Such as signal processing, or addressing your question, the SD card. If double buffering is used, once one buffer is complete, the data collection is then moved to the new buffer. The just completed buffer is then written to the SD card. And should complete in plenty of time before the next buffer is ready.

As the SD writing is occurring, samples coming in are handled on an interrupt basis. Interrupting very briefly the SD card writing routines. For another angle, @yj on this thread,

http://openbci.com/forum/index.php?p=/discussion/546/solution-for-session-freezing-overruns-2khz-sample-rate-with-sd-only

Took another approach, which was to not use the interrupts, but instead to sprinkle ADS1299 ready checks throughout the code of the SD driver. Either approach will work. It's my hunch that the interrupt based approach may be more generalized and will work with any time consuming operation without requiring code modification.

William

yj

@wjcroft & @ntlanh, & ...

yes interrupts is the best approach.

I hesitated to use them because I was afraid to break some time dependent processes during SD card writing...

For exemple is it possible to interrupt and release the SPI bus during any step of the 512 multiblocks writing?
It is out of my competence ...
Another exemple, one can read in the ADS1299 data sheet that the fact to release the spi bus re-initializes the communication process, is it more or less the same in some cases for the SD card ?
By principle the SPI bus is fully interruptable but not all the devices that use it... it's a pity...

( I or you, could try , there is not a lot to do from the pooling approach code, as soon I have time I'll try to see if the card  explodes)

If the new card called "ganglion" uses the 2 spi bus of the pic32 to dedicate one of them to the SD card, then it would be very efficient to use interrupts.

A last point : into some SD card (at least the one I use ) internal managment is such that , time to time, to write is not enable for rather long period of time (150mS on mine) then at "hight" sample rate , lets say 16k,  we need big circular or "double buffer" to not to lost the data.

  At least : 16x150x32 octes (and double if daisy)...

    Sorry for my english...

        Yannick.

biomurph

We will have an update to the firmware soon that will incorporate the interrupt for monitoring the ADS data ready pin.

yj

@biomurph

Good news,

what has been your motivation ?

- Overruns ellimination?
- Sample rate increase?
- Background task?
- ... ?

In any case ,  a big thank !!!

  Yannick.

biomurph

@hitheshaum

I have been looking at your code, and also digging into the interrupt documents from chipKIT. I don't quite understand, because the Board_Defs.h file specifies a different pin for the _EXTERNAL_4_VECTOR, but I have also implemented interrupts for monitoring the ADS DRDY pin. The code linked below only sets a flag in the ISR. Reading the data from the ADS may be coming soon. 

https://github.com/biomurph/OpenBCI_32_Interrupt

We are planning a firmware upgrade release this summer. PushTheWorld has been spearheading the work on this. You can see our progress here https://github.com/pushtheworldllc?tab=repositories

qwer1304

@biomurph

Do you use / intend to use DMA to get data off the ADS1299?

biomurph

@qwer1304, are you referring to Direct Memory Access?

I'm not quite sure how that would be implemented. I do know that it is important to do as little as possible inside an ISR, and things require timing may run into issues. We will be testing an ISR that reads the new ADS data, and stores it in a buffer (array), then sets a flag that the data is available to the rest of the program... Tho this scheme would need to practice some kind of double buffering, in case the main program is in the middle of accessing the data buffer... At the very least, with interrupts employed, the main loop can be free to do what it likes (DSP, for example) without the need to worry regarding finding the onset of the next available data packet.

qwer1304

@biomurph

Yes, that's Direct Memory Access.

The way it usually works is that in the ISR (which is triggered by Data Ready) you setup the DMA registers and release it (software control). Alternatively, you could set it up so that the demand interrupt (Data Ready) would automatically trigger the transfer (which would have been set up previously). The actual copying (transfer) of data is done by the HW (the DMA). This saves you the cycles for copying of data. You also get an interrupt on copying done.

The PIC32 has a DMA controller with 4 channels and can even do DMA between SPI and memory.

Take a look at Chapter 9 in the Datasheet and http://ww1.microchip.com/downloads/en/DeviceDoc/60001117H.pdf