Hi this article will show another work done by Chip, a great guy who has a very strong handling ablitity and profund knowledge , considering manyof you may come from China, this article will be translated in to chinese:

Ever since my effort with OpenBCI began, I’ve been looking to control something with my brain.  Sure, a while back, I was successful in lighting an LED with my brain waves, but that’s pretty simple.  I wanted something more.  And now I can do it.  I can control a robot with my mind!  Yes!

自从在BCI发力,我一直想用脑电控控制点什么,上次点亮LED太简单,这次哥要玩大的 控制机器人

Approach: 

My robot has just a few actions that it can do…turn left, turn right, walk forward, and fire.  To make this brain-controlled, I need a way to invoke these commands using signals from my brain.  Ideally, I’d just think the word “Fire!” and the robot would respond.  Unfortunately, those kinds of brain waves are too hard to detect.  Instead, I need to use brain waves that are easy to detect.  For me, “easy” brain waves include the Alpha waves (10 Hz oscillations) that occur when I close my eyes, as well as the brain waves that occur when I watch my blinking movies (a.k.a. visual entrainment).  So, my approach is to use OpenBCI to record my brainwaves, to write software to detect these specific types of brain waves, and to issue commands to the robot based on which brain waves are detected.

我的机器人只能 向左向右 向前和fire, 我必须用脑电信号控制机器,比如我想到 fire的时候 机器会给我反应,实际上这是很难得  因为无法探测到 想这个词的时候对应的脑电

简单的来看 可以用闭眼aphla(10HZ),还有观看闪烁的屏幕产生的脑电,然后利用这些信号 写点程序 就可以发送到机器端来控制啦

Hardware Setup: 

The core hardware for this hack is similar to my usual OpenBCI setup: EEG electrodes, an OpenBCI board, an Arduino Uno, and my computer. Added to this setup is the Hex Bug itself and its remote control, which I hacked so that the remote can be controlled by an Arduino.  So, as shown below, my brain wave signals go from my head all the way to the PC.  The PC processes the EEG data looking for the Alpha waves or the visually-entrained waves.  If any are detected, it decides what commands to give the robot.  The commands are conveyed back to the Arduino, which then drives the remote control, which the Hex Bug receives over its usual IR link.

硬件配置:

核心硬件配置主要是 电极  电板 Arduino等 另外需要设置hexbug的远程控制端 能够识别我发出的命令,PC 讲话处理记录的信号,如果发现设定的信号,及传送到通过IR连接的hexbug远程控制端

 

 

 

EEG Setup: 

I’m going to be measuring my Alpha waves and I’m going to be measuring the brain waves induced through visual entrainment.  Based on my previous experience, I know that both are best recorded using an electrode on the back of the head (at the “O1″ position, if you’re into your 10-20 electrode placement standard).  I do not need electrodes all over my head.  That’s the only sensing electrode that I’m using.  That’s it.  Of course, EEG also requires a reference electrode, which I put on my left earlobe.  And, finally, EEG often has a third electrode (“bias” or “driven ground”), which I placed on my right earlobe.

 

EEG设置

我需要识别Aphla波,根据之前的经验,视觉信号一般位于O1和O2位置,这两个电极所采集的信号基本够用啦

另外 参考电极设定在左耳垂,接地电极在有耳垂

 

Looking at the Frequency of my Brain Waves: 

As mentioned above, my approach is to control my robot by detecting Alpha waves and by detecting visually-entrained brain waves.  These are easily detectable because they occur at specific frequencies.  Alpha occur around 10 Hz and the visually-entrained brain waves occur at the blink rate(s) of whatever movies I use (my best results were from 5 Hz and 7.5 Hz movies). So, to control my robot, I will be looking for EEG signals at these frequencies: 5 Hz, 7.5 Hz, and 10 Hz.  I’m going to “look” for these frequencies by writing some EEG processing software that’ll look at the frequency content of my EEG signal to see if these frequencies are present

用获取的信号控制hexbug, 我们知道闭眼和眨眼产生的频率,所以我找到5 7.5 10hz来查看,

如上图,对获取的信号就行傅里叶变换得到频率,然后找到最大值和平均值,然后计算SNR 最后发出命令

 

EEG Processing: 

The flow chart above shows the steps that I use to process the EEG signal (my software is here).  Once the PC gets EEG data from the OpenBCI board, the first step is to compute the spectrum of the signal, which tells me the content of the EEG signal as a function of frequency.  I then search through the relevant part of the spectrum (4-15 Hz) to find the peak value.  I note both its frequency value and its amplitude.  In parallel, I also compute the average EEG amplitude across the 4-15Hz frequency band.  This average value is my baseline for deciding whether my peak is tall (strong) or short (weak).  By dividing the amplitude of my peak by this baseline value,  I get the signal-to-noise ratio (SNR) of the peak.  The SNR is my measure of the strength of the peak.  The output of the EEG processing, therefore, are two values: the frequency of the peak and the SNR of the peak.

 

EEG处理

PC一旦得到EEG信号,就会做傅里叶变换得到频率,我先找到4-15hz内的最大值,同步计算的平均值,这个作为准线来看看是否峰值的强弱,通过区分波峰的振幅,我可以得到波峰的信噪比,

信噪比是来衡量信号的强弱, 通过EEG 输出的有波峰的频率和信噪比

Deciding My Robot’s Action:  Once my EEG processing finds the frequency and SNR of the peak in my EEG spectrum, I now have to decide how to act on that information.  After some trial and error, I settled on the algorithm shown in the flow chart above.  It’s got three steps:

得到频率和信噪比,下面就需要来实现如何控制hexbug的行动

  • SNR Check:  First, I decide whether the current peak in the spectrum is legitimate, or if it is likely to be just noise.  I don’t want to issue a command if it is just noise because then my robot will be taking all sorts of actions that I didn’t intend.  That is not what I want.  So, to decide if the peak is likely to be legitimate, I look at the SNR of the peak.  If it has a big SNR, I’ll accept it as a legitimate peak.  If it is too small, I’ll take no further action.  Right now, my threshold for this decision is at 6 dB.  Setting a higher threshold results in fewer false commands (which would be good), but it also makes the system less sensitive to legitimate commands (which is bad).  This 6 dB threshold resulted in an OK (but not great) balance.

检查信噪比,首先要检查信噪比是否在范围内,我不想因为噪音 导致机器乱发命令,所以检查峰值是否在合理范围内,主要还是参考信噪比,如果太小  则不做任何动作,目前阀值是6DB,设置搞得阀值 可以减少误报率,问题是太高则系统的灵敏性大大降低。

  • Frequency Check:  If the peak seems legitimate, I decide how to command the robot based on the frequency of the peak.  If the peak is between 4.5-6.5 Hz, I must be looking at the right-side of my 2-speed blinking movie (ie, the portion that blinks at 5 Hz), so the computer prepares the “Turn Right” command.  Alternatively, if the EEG peak is 6.5-8.5 Hz, I must be looking at the left-side of my 2-speed blinking movie (ie, the portion that blinks at 7.5 Hz), so it prepares the “Turn Left” command.  Finally, if the EEG peak is 8.5-12 Hz, it must be my eyes-closed Alpha waves, so the computer prepares the “Move Forward” command.

如果频率在4.5-6.5hz 那么应该是向左看,那么系统就发出向左的命令,如果在6.5-8.5 那么我应该是向右看,系统就会发出向右的命令,如果在8.5-12这是闭眼产生的aphla波,对应向前的命令

  • New Command Check:  Before issuing the command, I check to see whether this command is the same as the last command that was extracted from my brain waves.  If the latest command is different, I hijack the command and, instead, issue the “Fire!” command.  If the latest command is the same, I go ahead and issue the left / right / forward command like normal.  The reason for this hijack is that I have no other type of easily-detected brain wave that I can use for commanding the robot to fire.  This approach of issuing “Fire!” on every change in command seemed like a decent way of getting a 4th command out of 3 types of brain waves.

在发出命令前,我需要检测这个命令是不是和上次执行的命令一样,如果不行 那么必须hijack的命令 发出fire的命令,如果一样 那么就按照正常执行,为什么这么做那? 因为无法检测到一个单独的命令可以识别fire,这个方案看起来是一种正常的方法,使用三种信号控制四种动作

 

Putting It All Together: 

 

As you can see in the movie, I eventually able to get all of these pieces working together to allow me to command the Hex Bug using just my brain waves.  Of course, it didn’t work the first time.  Even once I got all the hardware working, I still needed to tune a bunch of the software parameters (FFT parameters and the detection threshold) until I got something that worked somewhat reliably.  To help with this tuning process, I used the spectrum display that is in my Processing GUI.  Some screen shots are below.

 

最终看到 我将这些拼接在一起成功控制了hexbug,当然前几次是不成功,一旦所以硬件部分成功,我依然需要调节软件参数,就是傅里叶参数,这里主要利用频谱分布来实现

Example EEG spectrum when I stared at the right side of my two-speed blinking
movie.  It induced 5 Hz brain waves.  I programmed 5 Hz to mean “Turn Right”.
The SNR here is between 6 and 7 dB.

 

Here’s an example EEG spectrum when I stared at the left side of my two-speed
blinking movie.  It induced 7.5 Hz brain waves.  When the GUI detected 7.5 Hz,
it issued a “Turn Left” command to the Hex Bug.  The SNR is only 6-7 dB.

 

Finally, here’s an example EEG spectrum with my eyes closed so that I was
exhibiting Alpha waves, which are near 10 Hz.  When it detected 10 Hz, I
programmed it to issue a”Forward” command.  The SNR is > 8 dB.

 

 

Weak Peaks:

 

In the screenshots above, the red line shows the current EEG spectrum.  The heavy black circle shows the spectral peak that my software algorithms have detected.  The black dashed line is the “background noise” from which the SNR is computed.  To be declared a legitimate detection, the peak must be 6 dB higher than the black dashed line (unfortunately, I don’t show this on the plot…sorry!).  As can be seen, the 5 Hz and 7.5 Hz examples are not very strong (the SNR is only 6-7 dB).  Other peaks within the plots are very close to being the same size, which would cause false commands to be sent to the robot.  In my movie at the top of this post, there were several false commands.

上述 截图,红色显示EEG频率,黑色圈是频率的峰值,虚线是噪音,用来计算信噪比的, 可以看出频率比较接近,误报率很高
Balancing Sensitivity with False Commands:

To reduce the number of false commands, I could raise my detection threshold above 6 dB. Unfortunately, as see in the first two spectrum plots above, my 5 Hz and 7.5 Hz peaks are usually pretty weak (<  7 dB).  Therefore, any attempt to raise my detection threshold above 6 dB would cause me to no longer detect my legitimate brain waves.  I know because this is exactly the tuning process that I tried.  Bummer!  So, if I want more reliable performance, I’ll need to develop a fancier signal processing beyond this simple FFT-threshold approach.  Future challenges!

减少误报率 必须提高信噪比,5-7hz信号很微弱,提高阀值导致检测不到脑部信号,所以如果想得到理想的效果,简单的fft是不够的。

 
Wrapping Up:

Even with the false commands seen in my movie, I was still able to command the robot to move around the table.  I could get it to go (roughly) where I wanted it to go.  And, I did it all with just my brain waves.  I think that this is pretty exciting!  Yay!  What are the next steps?  Well, maybe now that I have this under my belt, I can move on to control flying fish, or maybe aquadcopter!  Do you have any other cool ideas for things I can control with my brain?

 

虽然问题很多,但是我还是成功控制了,这已经令人很兴奋,下一步可能控制飞鱼,直升机 或者其他cool cool的东西

 

Comments:

ya, this works sometime , so many weak points in this process, pattern recognization and analysis are undergoing problems for BCI,