LSL stream vs GUI / noise spectrum [resolved]

ht_steve1976ht_steve1976 Japan
edited March 2020 in Software

I would like to know the details of time series data on LSL.
I stream time series datas to MATLAB from OpenBCI_GUI. Then I applied the pre-processing code to online time series data on LSL as well as offline data from OpenBCI_GUI which were recorded automatically on that software.
But that didn't work well because those(time series data on LSL and time series data from OpenBCI_GUI) were not same.
And the spectrum of time series data on LSL are attached. That's wired.

So I'd like to know the details of time series data on LSL and the difference between the data on LSL and the data from OpenBCI_GUI.
If there are some information already, I'm sorry, but please tell me...

Thank you for your reading and cooperation.

Comments

  • ht_steve1976ht_steve1976 Japan
    edited January 2020

    In any cases, I would like to attach my MATLAB code here.

    %% obci_onPro_spectrum_t01.m
%% instantiate the library
disp('Loading the library...');
lib = lsl_loadlib();

disp(lib);

% resolve a stream...
disp('Resolving an EEG stream...');
result = {};
while isempty(result)
    result = lsl_resolve_byprop(lib,'type','EEG'); end

% create a new inlet
disp('Opening an inlet...');
inlet = lsl_inlet(result{1});

% disp(result);
% disp(inlet);

disp('Now receiving data...');

i=1;
L = 1000;    % 区間の点数？
Fs = 250;   % サンプリング周波数

MI = 10000;

eeg = zeros(MI,1);
eeg_st = zeros(L,1);
t_eeg = zeros(MI,1);

% サンプリング周波数確認用 1/2
freq = 0.0;
ptoc = 0.0;
j = 0;

% ファイルへの出力用
fileID = fopen('obci_onPro_t01.csv', 'w');
fprintf(fileID,'%6s %12s\n','ave','ts');

t = 1:500;

%% リアルタイムスペクトル表示
f = [1 2];
power1 = [1 2];

% p1 = plot(f, power1);
p1 = semilogy(f, power1);

xlim([0 125]);
ylim([0 1000]);
xlabel('frequency');
ylabel('power');

p1.XDataSource = 'f';
p1.YDataSource = 'power1';

X_theta47 = 1:250;

%%
for i=1:MI
    % get data from the inlet
    [vec,ts] = inlet.pull_sample();

    eeg(i) = vec(2);
    t_eeg(i) = ts;

    if i>L && rem(i,4) ==0

        eeg_st = eeg(i-L+1:i);

        k = rem(i,L)+1;

        if k == 5
            y1 = eeg_st;

            x1 = y1;
            y1 = fft(x1);
            n1 = length(x1);          % number of samples
            f = (0:n1-1)*(Fs/n1);     % frequency range
            power1 = abs(y1).^2/n1;    % power of the DFT

            axis auto;
            refreshdata;
            xlim([0 125]);
            drawnow;
        end

        fprintf(fileID,'%5.5f ',vec);
        fprintf(fileID,'%5.5f\n',ts);
    end
end
  • retiututretiutut Louisiana, USA
    edited January 2020

    The data in the Time Series widget is filtered if the filters are turned on, and you might be sending unfiltered data out from the networking widget.
    https://docs.openbci.com/docs/06Software/01-OpenBCISoftware/GUIWidgets#networking

  • ht_steve1976ht_steve1976 Japan

    Thank you very much for your comments.
    I understand that the data in the time series widget is filtered, but I don't get 25Hz noise if I turned off the notch filter and bandpass filter. Is there any other filter which I can not turn off ?

  • wjcroftwjcroft Mount Shasta, CA
    edited January 2020

    The image at the top of this thread shows a huge amount of 50 Hz mains noise as well as subharmonics and harmonics, for example the 25 Hz and harmonics of that. A single notch at 50 Hz will probably kill most of it, including harmonics.

  • ht_steve1976ht_steve1976 Japan

    @wjcroft
    Thank you very much for your advice. Sorry for this late comment.
    I will apply notch filter at 50Hz to my data and comment again.
    But it's little wired for me because our area's power supply frequency is 60Hz. If you know some reasons, please let me know...
    And there are two large noise which the frequencies are lower than 25Hz. It's also wired because the data from OpenBCI_GUI does not have those.
    I will make the power spectrum to compare the data from OpenBCI_GUI and MATLAB which is sent by LSL function and post again.

  • wjcroftwjcroft Mount Shasta, CA

    Harmonics and sub-harmonics can produce unusual frequency spectra. The top graph 'appears' to have both 60 and 50 components. But as you say the 60 Hz fundamental is there prominently. I would just start with the notch at your mains frequency, 60 Hz. And with a relatively quiet EEG, these harmonics will go away. If you see another large spike after the 60 Hz notch is in place, try another notch at that noise source frequency.

    The spikes below 20 Hz are clearly part of the same harmonic pattern present in the rest of the spectra. Kill the root cause and the harmonics should disappear.

    If you have a large noise spike that is NOT a mains frequency, certainly also look into why that is present in your environment, and try to reduce or eliminate it at the source.

  • ht_steve1976ht_steve1976 Japan

    @wjcroft
    Thank you very much for your kind comment.
    Recently, I tried 50Hz notch filter to the data from MATLAB. But I think the data is not good enough. I attached the image. Blue line showed the data from MATLAB (no filter), Orange line showed the data from MATLAB (50Hz notch filter), Yellow line showed the data from OpenBCI_GUI. I also tried 25Hz notch filter to the data, but the result is almost similar, only 25Hz noise decreased.
    I will search the method of notch filter in MALTAB again.

    I still wonder why the data from OpenBCI_GUI and the data from MATLAB are so different. (The data from MATLAB means the data which MATLAB get though LSL function of OpenBCI_GUI.) I think if the problem was my environment, the data from OpenBCI_GUI would have same noise. Is this correct?

  • wjcroftwjcroft Mount Shasta, CA

    @wjcroft said:
    Harmonics and sub-harmonics can produce unusual frequency spectra. The top graph 'appears' to have both 60 and 50 components. But as you say the 60 Hz fundamental is there prominently. I would just start with the notch at your mains frequency, 60 Hz. And with a relatively quiet EEG, these harmonics will go away. If you see another large spike after the 60 Hz notch is in place, try another notch at that noise source frequency.

    I thought you previously said your mains frequency was 60 Hz. You have to start with the 60 Hz notch, as I mentioned.

  • ht_steve1976ht_steve1976 Japan

    @wjcroft
    Thank you for your comment.
    I applied 60Hz and 50Hz notch filter. But the data from MATLAB(orange line) and the data from OpenBCI_GUI(yellow line) are so different.
    Is there anyone who got the same issue...?

  • wjcroftwjcroft Mount Shasta, CA

    Is it possible there is something wrong with the way you are pulling the data stream from LSL? The FFT for the GUI csv looks ok, with many various small peaks throughout the spectrum; and the expected 60 Hz mains larger peak. Your FFT of the LSL stream looks completely different. So one would assume the LSL time series stream is distorted. Have you tried plotting the time series from the LSL versus the time series from CSV? That could point out what is going wrong.

    Likely this whole diversion into discussion about notch filters was off the track. Your initial post in this thread mentions that "...But that didn't work well because those(time series data on LSL and time series data from OpenBCI_GUI) were not same..." That is where the real issue is.

    Do note that the time series shown on the GUI generally has had several filters applied already. Even when "turned off" there is still some high pass filtering going on at say .5 Hz, to remove the DC offset from Cyton. So be sure to incorporate high pass if you are comparing time series.

  • MrKlean518MrKlean518 Las Vegas

    Hello, I think I know what is going on. Based on the data being several orders of magnitude larger at certain points, I believe the issue is that if you are not using their new BrainFlow API, you are representing the ADC values before the conversion to uV. As per the pyOpenBCI Documentation the signals coming in from the board need to be multiplied by a constant (4500000/24)/((2^23)-1) to convert the values to uV. THEN the filtering should be performed.

  • wjcroftwjcroft Mount Shasta, CA

    @MrKlean518, thanks. But the LSL stream does not contain raw ADC values. It has been converted already.

    Regards, William

  • MrKlean518MrKlean518 Las Vegas

    Oh I apologize. Is that unique to streaming over LSL from the GUI? In the example given for LSL streaming in the repo, it still processes the LSL stream through the constant?

  • wjcroftwjcroft Mount Shasta, CA

    Steve said in the first post on this thread, that he is streaming the LSL from the GUI, not from Python. In general, data on an LSL stream should already be in standardized units, not raw ADC values.

    You are correct in that previous Python based LSL streaming is phasing out, in favor of BrainFlow library. As far as I am aware, there is yet no BrainFlow for Matlab, which is the language / environment that Steve is consuming the data stream in. And why he is using LSL as intermediary.

    Regards, William

  • MrKlean518MrKlean518 Las Vegas

    Yeah I read that he is streaming over the GUI, hence why I was asked if it was unique to it. I just assumed that the LSL stream was the same regardless of the streaming source as long as it came from the same board, that was my mistake. Im genuinely curious in trying to find a solution though, as I have been pretty heavily engrossed in EEG pre-processing the last month or so. Whenever I was using the GUI with the BLE dongle, i noticed a lot of artifacts/noise from dropped packets. Does LSL alleviate the dropped packet issue?

    One small question, because he is processing data recorded and saved via the GUI offline, is adding the high-pass/notch filter necessary? If I recall (i might be wrong here) the GUI displays filtered data, but still saves unfiltered raw uV values to the text file. Or am I wrong, and the DC offset is removed even in the text file?

  • wjcroftwjcroft Mount Shasta, CA

    The GUI CSV file is unfiltered. If you are seeing dropped packets with the Cyton dongle, it's generally because of a noisy microwave radio environment in your area. You can change the radio channel to see if that improves. Another angle is to use a usb extension cable to get the dongle closer to the mainboard. Recent GUI improvements have added 'interpolation' to reduce the impact of lost packets. You could do the same in your software.

  • MrKlean518MrKlean518 Las Vegas

    Sorry, I should have clarified, I am not asking in regards to my own software as I am currently using the BrainFlow API which seems to alleviate the dropped packets issue. I was just asking general questions, thinking in respect to Steve's post. But if that's too off topic for this no worries.

  • wjcroftwjcroft Mount Shasta, CA

    BrainFlow is likely doing the same interpolation for lost packets. In Steve's case he is receiving the LSL stream from the GUI, which should include interpolation. Other threads on the forum mention dropped packets, available with the search button.

  • ht_steve1976ht_steve1976 Japan

    @MrKlean518 @wjcroft
    Thank you very much for your comments.
    Recently, I tried the same Matlab program in another Mac. In that Mac, this problem didn't happen...
    I got the spectrum which seem Okey. It's similar the spectrum in OpenBCI_GUI...
    I don't know clearly, but I think this problem is my Mac's hardware problem.
    I tried to restart Mac and finished other programs, but the problem was still there.

    I will post the spectrum which I got in another Mac later.

    In any cases, the specification of my Mac is below.
    MacBook Pro (Retina, 13-inch, Early 2015)
    2.7 GHz Dual Core Intel Core i5
    16 GB 1867 MHz DDR3
    Intel Iris Graphics 6100 1536 MB

  • ht_steve1976ht_steve1976 Japan

    I finally understood why.
    I misunderstood about the Filters button. I thought 1-ON button means Stream1 will be ON, but that's wrong. I checked other protocols (Serial, UDP, OSC), and I understood 1-ON, 1-OFF are about Filters.
    Thank you very much for your cooperation.

  • wjcroftwjcroft Mount Shasta, CA

    Steve, thanks.

    And sorry about the missing label 'Filters' for the LSL widget. It is present for all the other Networking widgets. I guess the label just got left out for some reason,

    https://docs.openbci.com/docs/06Software/01-OpenBCISoftware/GUIWidgets#networking

    Mentioning Richard @retiutut.

    Regards, William

  • retiututretiutut Louisiana, USA

    Will do!

  • retiututretiutut Louisiana, USA

    Done! https://github.com/OpenBCI/OpenBCI_GUI/issues/696

  • ht_steve1976ht_steve1976 Japan

    Thank you very much for continuous helps and immediate revision!!
    Now I don't have any problem.

Sign In or Register to comment.

© Vanilla Keystone Theme 2026