This project will show you how take a Radio Controller Helicopter and
modify the remote control hardware such that it can be operated by free, open source computer software and flown based on brainwave measurements of concentration and relaxation taken by consumer-grade EEG headsets.
The software used in this Instructable consists of two applications, Puzzlebox Synapse and Puzzlebox Brainstorms . The former connects to commercially available consumer-grade EEGheadsets, such as the NeuroSky MindSet or Emotiv EPOC . The latter connects to the transmitter chip extracted from the RC Helicopter's remote control and issues flight commands and settings based on detections received from Puzzlebox Synapse .
Download Source Codes given below:
Source Code 1
Source Code 2
Required Materials
Note : Items in bold can be found in the Glossary and Link Index listed the final step of this Project.
Before beginning, unpack the helicopter, charge and install the batteries, and make sure everything is in good working order before beginning to examine or disassemble any individual components.
You should also familiarize yourself with the basic controls of your helicopter, including how trim settings operate and basic flying characteristics.
Note: The RC Helicopter used is a Blade mCX2 :
http://www.bladehelis.com/Products/Default.aspx?ProdId=EFLH2400
Note: The remote control used for this Instructable is a Blade MLP4DSM .
There are 4 channels which are used for flying:
The two joysticks are connected to potentiometers which adjust the voltage of the circuit to which each are connected from the DC power source through to an Atmega88PA micro-controller.
The micro-controller measures the input voltages and converts them to digital PCM values which are sent to a transmitter chip which uses the Spektrum DSM2 protocol to communicate with the RC Helicopter .
Layman's Explanation:
(Note: this explanation is not 100% accurate but may be a helpful way to visualize and understand what is going on)
It may be helpful to think of electricity as water, flowing through pipes as opposed to wires or tracks on the circuit board. If you were to measure the pressure or level of the water flowing inside the pipe at any one point that would be its voltage , with the diameter of the pipe the maximum voltage the circuit can handle. The volume of the water passing through that point is the current of the circuit, measured in amperes (amps). If you were to picture a large storm drain dripping a trickle of water that would be a high voltage, low current circuit, whereas a fire hose blasting water would be a relatively low voltage, high current circuit. It is for this reason that amps are often considered more dangerous than voltage.
The batteries act as the both the source and the destination for the water (electricity) flowing through the pipes (circuit). The potentiometers connected to the joysticks on the front of the remote control act as gates which alter the level (voltage ) of water flowing through the pipes. For example, when the throttle joystick is in the lowest position, where the rotor blades are normally off, the gate is entirely closes so no water is flowing throw the pipes and the voltage is zero. When the the throttle is in the highest position the gate is completely open and the water and voltage are likewise at their highest level.
If the pipes passed through a wheel on the way to their destination and that wheel turned the helicopter rotor blades, it might make sense that more water flowing through (high voltage ) would turn the wheel and therefore rotor blades faster, allowing the helicopter to fly higher.
Now of course the joysticks and circuit we are talking about is on the remote control, not the helicopter, so we need a few extra steps.
In this imaginary example, the micro-controller measures the water level in the pipe as it flows past. This number represents the voltage and therefore the joystick position. The number is converted into a digital format and passed on to the transmitter, along with the number which correspond to all of the other joystick positions. The transmitter broadcasts these numbers to the helicopter. Finally the receiver in the helicopter talks to its own micro-controller which converts the digital values into the appropriate physical settings for the rotor blades and servos.
In this example video the Radio Controlled Helicopter is first operated via the original remote control, then after switches on the prototype board are flipped, the command to "fly forward" is issued by thePuzzlebox Brainstorms software operating in console mode. Note there is a delay before the helicopter takes off during which time the sync between transmitter and helicopter is re-established.
Available console commands include the following:
Neutral: Use this command to establish initial sync with helicopter
python Helicopter_Control.py --command=neutral
Hover: Hover the helicopter in the air
python Helicopter_Control.py --command=hover
Fly Forward: Fly the helicopter forward, low to the ground so as to easily land
python Helicopter_Control.py --command=fly_forward
Read: Read data frames from the remote control and output them directly to the console
python Helicopter_Control.py --command=read
modify the remote control hardware such that it can be operated by free, open source computer software and flown based on brainwave measurements of concentration and relaxation taken by consumer-grade EEG headsets.
The software used in this Instructable consists of two applications, Puzzlebox Synapse and Puzzlebox Brainstorms . The former connects to commercially available consumer-grade EEGheadsets, such as the NeuroSky MindSet or Emotiv EPOC . The latter connects to the transmitter chip extracted from the RC Helicopter's remote control and issues flight commands and settings based on detections received from Puzzlebox Synapse .
Download Source Codes given below:
Source Code 1
Source Code 2
Required Materials
- Radio Controlled Helicopter
- EEG headset such as the NeuroSky MindSet or Emotiv EPOC
- USB-to-Serial converter cable, capable of being set at an arbitrary baud rate
- An oscilloscope
- A logic analyzer
- Soldering Iron and Solder (optional)
- Connection cables and prototyping board (optional)
- Puzzlebox Synapse and Puzzlebox Brainstorms software
Note : Items in bold can be found in the Glossary and Link Index listed the final step of this Project.
Before beginning, unpack the helicopter, charge and install the batteries, and make sure everything is in good working order before beginning to examine or disassemble any individual components.
You should also familiarize yourself with the basic controls of your helicopter, including how trim settings operate and basic flying characteristics.
Note: The RC Helicopter used is a Blade mCX2 :
http://www.bladehelis.com/Products/Default.aspx?ProdId=EFLH2400
Step 1Examine Remote Control Components
The first step is to disassemble and examine the components of the circuit board of the remote control. It may be useful to employ a microscope to examine the manufacturer's name and part number of the various chips in order to research their features and protocols. This can save time later when working with the oscilloscope and logic analyzer .
Make your own Oscilloscope and Logic analyer
Note: The remote control used for this Instructable is a Blade MLP4DSM .
There are 4 channels which are used for flying:
- Throttle (Up/Down in the air)
- Elevator (Forward/Backward while flying)
- Aileron (Left/Right while flying)
- Rudder (Rotate Clockwise/Counter-clockwise while flying)
The two joysticks are connected to potentiometers which adjust the voltage of the circuit to which each are connected from the DC power source through to an Atmega88PA micro-controller.
The micro-controller measures the input voltages and converts them to digital PCM values which are sent to a transmitter chip which uses the Spektrum DSM2 protocol to communicate with the RC Helicopter .
Layman's Explanation:
(Note: this explanation is not 100% accurate but may be a helpful way to visualize and understand what is going on)
It may be helpful to think of electricity as water, flowing through pipes as opposed to wires or tracks on the circuit board. If you were to measure the pressure or level of the water flowing inside the pipe at any one point that would be its voltage , with the diameter of the pipe the maximum voltage the circuit can handle. The volume of the water passing through that point is the current of the circuit, measured in amperes (amps). If you were to picture a large storm drain dripping a trickle of water that would be a high voltage, low current circuit, whereas a fire hose blasting water would be a relatively low voltage, high current circuit. It is for this reason that amps are often considered more dangerous than voltage.
The batteries act as the both the source and the destination for the water (electricity) flowing through the pipes (circuit). The potentiometers connected to the joysticks on the front of the remote control act as gates which alter the level (voltage ) of water flowing through the pipes. For example, when the throttle joystick is in the lowest position, where the rotor blades are normally off, the gate is entirely closes so no water is flowing throw the pipes and the voltage is zero. When the the throttle is in the highest position the gate is completely open and the water and voltage are likewise at their highest level.
If the pipes passed through a wheel on the way to their destination and that wheel turned the helicopter rotor blades, it might make sense that more water flowing through (high voltage ) would turn the wheel and therefore rotor blades faster, allowing the helicopter to fly higher.
Now of course the joysticks and circuit we are talking about is on the remote control, not the helicopter, so we need a few extra steps.
In this imaginary example, the micro-controller measures the water level in the pipe as it flows past. This number represents the voltage and therefore the joystick position. The number is converted into a digital format and passed on to the transmitter, along with the number which correspond to all of the other joystick positions. The transmitter broadcasts these numbers to the helicopter. Finally the receiver in the helicopter talks to its own micro-controller which converts the digital values into the appropriate physical settings for the rotor blades and servos.
Step 2Connect Control Board to Oscilloscope
By utilizing an oscilloscope , we can determine some of the characteristics of the signal sent between the micro-controller and the transmitter chip.
We connected the lead and ground wire to each of the pins of the transmitter chip until we were able to determine which pin was used for the signal and which was the electrical ground. Adjusting the intensity of the oscilloscope will help to make the image more clearly visible.
Most importantly we were able to determine that the remote control for our helicopter sends a digital as opposed to analog signal.
Also quite important, we were able to measure the voltage level used by the Digital signal to confirm we could read and generate a signal at the same voltage level using our USB-to-Serial cable. If this were not the case we might need to add a simple circuit to step up or down the circuit voltage to match our USB-to-Serial hardware.
In the first photo below the dots along the lower line indicate when packets of data (aka "frames") are being sent. We were able to time the impulses to get an idea of what we need to look at when we moved on to the logic analyzer .
In the second photo by zooming in we can see some of the verticals from the digital signal. Sections of the lines will flicker as the joysticks on the remote control's circuit board are manipulated. This indicates the data in each frame is changing in relation to the directions indicated by each channel.
More specific directions and discussion regards how to use an oscilloscope are beyond the scope of this document.
We connected the lead and ground wire to each of the pins of the transmitter chip until we were able to determine which pin was used for the signal and which was the electrical ground. Adjusting the intensity of the oscilloscope will help to make the image more clearly visible.
Most importantly we were able to determine that the remote control for our helicopter sends a digital as opposed to analog signal.
Also quite important, we were able to measure the voltage level used by the Digital signal to confirm we could read and generate a signal at the same voltage level using our USB-to-Serial cable. If this were not the case we might need to add a simple circuit to step up or down the circuit voltage to match our USB-to-Serial hardware.
In the first photo below the dots along the lower line indicate when packets of data (aka "frames") are being sent. We were able to time the impulses to get an idea of what we need to look at when we moved on to the logic analyzer .
In the second photo by zooming in we can see some of the verticals from the digital signal. Sections of the lines will flicker as the joysticks on the remote control's circuit board are manipulated. This indicates the data in each frame is changing in relation to the directions indicated by each channel.
More specific directions and discussion regards how to use an oscilloscope are beyond the scope of this document.
Step 3Connect Control Board to Logic Analyzer
A logic analyzer will allow you to capture the frames of digital PCM data as the signal is sent from the micro-controller to the transmitter and visualize and decode them.
For this Instructable first a Tektronix 1241 Logic Analyzer was used, then a Saleae Logic which allowed us to precisely measure the frequency with which frames of data were sent (22ms) including the format, quantity, and content of characters in each byte of its serial protocol (8 data bits, no parity, 1 stop bit), as well as its rate (133000 baud ). Each frame contains 14 bytes which can be conveniently represented in hex notation.
Note : The baud rate quite critical. The closest standard baud to our target supported by most if not all serial devices (including the USB-to-Serial device we require) is 115200. This was too far out from our transmitter and during initial experimentation attempts at communication failed. It became necessary to select a USB-to-Serial device whose chipset permitted the setting of an arbitrary baud rate. For the final version we settled on a model which included a FTDI chipset (specifically the FT232 USB-Serial (UART) IC, see Glossary for details).
Attached to this step are example log files captured with the "Logic" software application freely available from Saleae. In the first file the throttle is completely in the down position, in the second file the throttle is completely in the up position.
More specific directions and discussion regards how to use a logic analyzer are beyond the scope of this document.
Layman's Explanation :
(Note : this explanation is not 100% accurate but may be a helpful way to visualize and understand what is going on)
In our earlier analogy, we stated that the water pressure and level as it flowed through pipes was akin to the voltage level of the electrical current flowing through our circuit, being altered by the "gates" of the joysticks being raised and lowered. As we learned by using the oscilloscope however, the signal sent between the micro-controller and the transmitter chip is digital , not analog . This effectively means that any time we take a measurement the water pipe will either be full, or empty at any given point, never in between. By alternating between these two extremes, binary numbers are being sent.
Another way to think about this is to imagine two people at either end of the pipe, a sender and a receiver. In this case the sender is the micro-controller (because it knows the value of the numbers to be sent) and the receiver is the transmitter chip. Attempting to send information by flooding and draining the pipes would be very slow, so instead they are using a flashlight to communicate in a form of morse code .
Every so often, the receiver will peek into the pipe and at the same moment the sender will either turn on the flashlight to indicate a one, or leave it off to indicate a zero. Each time this happens a single bitis communicated. This will happen rapidly eight times in a row, forming a byte . If every bit was a letter in a word, the work would be the byte . The logic analyzer further tells us that there are fourteenbytes or "words" in each sentence spoken by the micro-controller to the transmitter chip, and each sentence contains all of the joystick settings for each motion direction the remote control is capable of setting. The complete sentence gets constantly repeated approximately 45 times per second (once every 22ms) as long as the remote control is powered on.
The baud rate mentioned above as being critical would be the predetermined timing the sender and received would have agreed upon for sending flashes of light. In other words if the sender is going to send a flash once every five seconds but the receiver was only looking once every ten seconds, the receiver would only be there to witness half of the signals, and after the first view would record each of the subsequent bits in the wrong positions within the words they are recording. When we say the baudrate is 133000 what that actually means is there are up to 133,000 individual flashes happening every single second!
For this Instructable first a Tektronix 1241 Logic Analyzer was used, then a Saleae Logic which allowed us to precisely measure the frequency with which frames of data were sent (22ms) including the format, quantity, and content of characters in each byte of its serial protocol (8 data bits, no parity, 1 stop bit), as well as its rate (133000 baud ). Each frame contains 14 bytes which can be conveniently represented in hex notation.
Note : The baud rate quite critical. The closest standard baud to our target supported by most if not all serial devices (including the USB-to-Serial device we require) is 115200. This was too far out from our transmitter and during initial experimentation attempts at communication failed. It became necessary to select a USB-to-Serial device whose chipset permitted the setting of an arbitrary baud rate. For the final version we settled on a model which included a FTDI chipset (specifically the FT232 USB-Serial (UART) IC, see Glossary for details).
Attached to this step are example log files captured with the "Logic" software application freely available from Saleae. In the first file the throttle is completely in the down position, in the second file the throttle is completely in the up position.
More specific directions and discussion regards how to use a logic analyzer are beyond the scope of this document.
Layman's Explanation :
(Note : this explanation is not 100% accurate but may be a helpful way to visualize and understand what is going on)
In our earlier analogy, we stated that the water pressure and level as it flowed through pipes was akin to the voltage level of the electrical current flowing through our circuit, being altered by the "gates" of the joysticks being raised and lowered. As we learned by using the oscilloscope however, the signal sent between the micro-controller and the transmitter chip is digital , not analog . This effectively means that any time we take a measurement the water pipe will either be full, or empty at any given point, never in between. By alternating between these two extremes, binary numbers are being sent.
Another way to think about this is to imagine two people at either end of the pipe, a sender and a receiver. In this case the sender is the micro-controller (because it knows the value of the numbers to be sent) and the receiver is the transmitter chip. Attempting to send information by flooding and draining the pipes would be very slow, so instead they are using a flashlight to communicate in a form of morse code .
Every so often, the receiver will peek into the pipe and at the same moment the sender will either turn on the flashlight to indicate a one, or leave it off to indicate a zero. Each time this happens a single bitis communicated. This will happen rapidly eight times in a row, forming a byte . If every bit was a letter in a word, the work would be the byte . The logic analyzer further tells us that there are fourteenbytes or "words" in each sentence spoken by the micro-controller to the transmitter chip, and each sentence contains all of the joystick settings for each motion direction the remote control is capable of setting. The complete sentence gets constantly repeated approximately 45 times per second (once every 22ms) as long as the remote control is powered on.
The baud rate mentioned above as being critical would be the predetermined timing the sender and received would have agreed upon for sending flashes of light. In other words if the sender is going to send a flash once every five seconds but the receiver was only looking once every ten seconds, the receiver would only be there to witness half of the signals, and after the first view would record each of the subsequent bits in the wrong positions within the words they are recording. When we say the baudrate is 133000 what that actually means is there are up to 133,000 individual flashes happening every single second!
Step 4Desolder Transmitter from Control Board
After using the oscilloscope to establish a digital signal was being sent between the micro-controller and the transmitter chip, and using the logic analyzer to decode the characteristics and content of its serial protocol, it became clear that the easiest method for us to interface with our radio controlled helicopter via software would be to bypass all other components and communicate with the transmitter chip direction using a USB-to-Serial cable.
Had this not been the case we might have instead used the analog outputs of an Arduino to simulate the voltage levels normally indicated by the potentiometers .
As the transmitter chip was actually a separate circuit board (see photo) and there was enough clearance on the pins connecting the two for us to clip in, we might have been able to record and analyze all of the data being sent to it from the micro-controller without any physically modifications. However we eventually would want to be able to simulate these messages ourselves, which would require powering the transmitter board. If the power switch of the remote control was on then the micro-controller would also be sending data, which would conflict with the data we were generating. If we left the power switch off the transmitter board would also be off, preventing it from sending any data to the helicopter. We considered that we would be able to power the transmitter circuit from the USB-to-Serial cable, however we were not certain that the electricity applied at this point would not also flow back to the micro-controller.
Therefore we decided to desolder the transmitter chip from the remote control entirely as that was the only component we required for certain for our project.
Had this not been the case we might have instead used the analog outputs of an Arduino to simulate the voltage levels normally indicated by the potentiometers .
As the transmitter chip was actually a separate circuit board (see photo) and there was enough clearance on the pins connecting the two for us to clip in, we might have been able to record and analyze all of the data being sent to it from the micro-controller without any physically modifications. However we eventually would want to be able to simulate these messages ourselves, which would require powering the transmitter board. If the power switch of the remote control was on then the micro-controller would also be sending data, which would conflict with the data we were generating. If we left the power switch off the transmitter board would also be off, preventing it from sending any data to the helicopter. We considered that we would be able to power the transmitter circuit from the USB-to-Serial cable, however we were not certain that the electricity applied at this point would not also flow back to the micro-controller.
Therefore we decided to desolder the transmitter chip from the remote control entirely as that was the only component we required for certain for our project.
Step 5Connect Control Board to USB/Serial Cable
Once the transmitter chip was desoldered, we wanted to be able to reconnect the original remote control to it and still fly the helicopter as before, or be able to connect the USB-to-Serial cable to it and operate the helicopter using software, or be able to connect the remote control to the USB-to-Serial cable to capture and record data frames directly.
We ended up building a simple switch circuit using a prototype board which would allow us to flip switches as desired into any of these configurations.
We used the information collected in this way to write in Python the "Helicopter_Control.py" module available as part of the Puzzlebox Brainstorms software.
In the attached photos we ran this module from a console, issuing the "read" command in the following way:
python Helicopter_Control.py --command=read --device=/dev/ttyUSB0
Note : In our specific configuration, the serial port created on our Linux system when the USB-to-Serial device was attached was "/dev/ttyUSB0" and this may be different for your specific system. Under Windows a COM port such as "COM1" might need to be indicated.
When executed in this way it becomes quite easy to see the realtime data sent from the micro-controller to the transmitter chip. We incorporated this information into our source code such that we could arbitrarily send specific command strings to our helicopter at will.
Incidentally we noticed that when the helicopter is first powered up, it is necessary for a command string matching a "neutral" throttle position to be repeated sent by either the remote control or computer software, whichever is currently connected to the transmitter chip. Otherwise the helicopter will not see the transmitter chip and will enter sync mode (indicated by a blinking blue light in the cockpit).
Also when the console command is terminated the helicopter will disconnect (the constant blue light in cockpit going out). To re-establish the connection it is necessary to again transmit a neutral signal for approximately two seconds before a directional command can be issued, such as "hover" or "fly forward." These transmission settings are handled automatically by the Puzzlebox Brainstorms.
We ended up building a simple switch circuit using a prototype board which would allow us to flip switches as desired into any of these configurations.
We used the information collected in this way to write in Python the "Helicopter_Control.py" module available as part of the Puzzlebox Brainstorms software.
In the attached photos we ran this module from a console, issuing the "read" command in the following way:
python Helicopter_Control.py --command=read --device=/dev/ttyUSB0
Note : In our specific configuration, the serial port created on our Linux system when the USB-to-Serial device was attached was "/dev/ttyUSB0" and this may be different for your specific system. Under Windows a COM port such as "COM1" might need to be indicated.
When executed in this way it becomes quite easy to see the realtime data sent from the micro-controller to the transmitter chip. We incorporated this information into our source code such that we could arbitrarily send specific command strings to our helicopter at will.
Incidentally we noticed that when the helicopter is first powered up, it is necessary for a command string matching a "neutral" throttle position to be repeated sent by either the remote control or computer software, whichever is currently connected to the transmitter chip. Otherwise the helicopter will not see the transmitter chip and will enter sync mode (indicated by a blinking blue light in the cockpit).
Also when the console command is terminated the helicopter will disconnect (the constant blue light in cockpit going out). To re-establish the connection it is necessary to again transmit a neutral signal for approximately two seconds before a directional command can be issued, such as "hover" or "fly forward." These transmission settings are handled automatically by the Puzzlebox Brainstorms.
Step 6Test Flying RC Heliocopter from Command Console
In this example video the Radio Controlled Helicopter is first operated via the original remote control, then after switches on the prototype board are flipped, the command to "fly forward" is issued by thePuzzlebox Brainstorms software operating in console mode. Note there is a delay before the helicopter takes off during which time the sync between transmitter and helicopter is re-established.
Available console commands include the following:
Neutral: Use this command to establish initial sync with helicopter
python Helicopter_Control.py --command=neutral
Hover: Hover the helicopter in the air
python Helicopter_Control.py --command=hover
Fly Forward: Fly the helicopter forward, low to the ground so as to easily land
python Helicopter_Control.py --command=fly_forward
Read: Read data frames from the remote control and output them directly to the console
python Helicopter_Control.py --command=read
Step 7Connect EEG Headset to Puzzlebox Synapse
Puzzlebox Synapse is a free, open source, cross-platform application which connects directly to commercially available consumer-grade EEG headsets, collects brainwaves signals (including detection states), performs visualizations of the data, optionally records sessions to disk, and provides a TCP/IP server infrastructure to relay information to remote clients.
For this Instructable we will use Puzzlebox Synapse to connect to a NeuroSky MindSet and report calculations of "attention" and "meditation" levels to Puzzlebox Brainstorms which uses this data to fly, hover, or land the Radio Controlled Helicopter .
If you are using an Emotiv EPOC headset then it is possible to use the "EmoKey" software to talk directly to Puzzlebox Brainstorms and thus skip this step.
To begin you will need to establish a Bluetooth connection with your NeuroSky MindSet . Instructions on how to do this should have been provided with your headset and are outside the scope of this document. That said if you are using Windows XP, it is recommended that you use the Toshiba Bluetooth stack (provided on the headset's installation CD) as opposed to the built-in Microsoft Bluetooth stack. We have simply had better results with Toshiba's software. The default Bluetooth pin for the NeuroSky MindSet is "0000" and once connected you should have a new COM port for your windows system or /dev/ttyUSB under Linux. Any users of Mac OS X are encouraged to try running the software from source using a Python interpreter and reporting any issues they might encounter. The software should be compatible but has neither been tested with nor packaged for OS X at the time of writing.
Once connected simply select the Bluetooth MAC address of your connected NeuroSky MindSet , or select the indicated COM port as appropriate.
Click the "Start" button under the "Server Daemon" section of the interface and the GUI will begin displaying EEG measurements in realtime. Brainwave and "eSense" calculations are produced once per second. It is also possible to view the user's raw EEG output as graphed waves, view attention and relaxation trends over time, and save the complete contents of a user's session to disk or export the results as a CSV file for use with OpenOffice.org Calc or Microsoft Excel.
For this Instructable we will use Puzzlebox Synapse to connect to a NeuroSky MindSet and report calculations of "attention" and "meditation" levels to Puzzlebox Brainstorms which uses this data to fly, hover, or land the Radio Controlled Helicopter .
If you are using an Emotiv EPOC headset then it is possible to use the "EmoKey" software to talk directly to Puzzlebox Brainstorms and thus skip this step.
To begin you will need to establish a Bluetooth connection with your NeuroSky MindSet . Instructions on how to do this should have been provided with your headset and are outside the scope of this document. That said if you are using Windows XP, it is recommended that you use the Toshiba Bluetooth stack (provided on the headset's installation CD) as opposed to the built-in Microsoft Bluetooth stack. We have simply had better results with Toshiba's software. The default Bluetooth pin for the NeuroSky MindSet is "0000" and once connected you should have a new COM port for your windows system or /dev/ttyUSB under Linux. Any users of Mac OS X are encouraged to try running the software from source using a Python interpreter and reporting any issues they might encounter. The software should be compatible but has neither been tested with nor packaged for OS X at the time of writing.
Once connected simply select the Bluetooth MAC address of your connected NeuroSky MindSet , or select the indicated COM port as appropriate.
Click the "Start" button under the "Server Daemon" section of the interface and the GUI will begin displaying EEG measurements in realtime. Brainwave and "eSense" calculations are produced once per second. It is also possible to view the user's raw EEG output as graphed waves, view attention and relaxation trends over time, and save the complete contents of a user's session to disk or export the results as a CSV file for use with OpenOffice.org Calc or Microsoft Excel.
Step 8Fly RC Helicopter Using Puzzlebox Brainstorms
Puzzlebox Brainstorms is a free, open source, cross-platform software application which permitsBrain-Computer Interface (BCI) control of vehicles, devices, and toys such as LEGO Mindstorms ,Radio Controlled Helicopters , and even electric wheelchairs.
In this Instructable we will use this software to fly our helicopter forward, hover, and/or land automatically based on detected levels of concentration and relaxation from an EEG headset such as the NeuroSky MindSet or Emotiv EPOC .
The first step, after loading the application, is to connect to the Puzzlebox Synapse server which was prepared in the previous step. The server can exist on the same computer system or be accessed remotely across a network or the Internet over TCP/IP. Click on the "Control Panel" tab of Puzzlebox Brainstorms and after verifying the Host and Port settings, click "Connect" to begin receiving EEGdetections.
Next click on the "RC Helicopter" tab.
The transmitter chip should have already been extracted and connected to the computer system as described in earlier steps. The USB-to-Serial interface will appear as a COM port under Windows or a /dev/ttyUSB serial device under Linux. By default, when "Concentration" or "Relaxation" levels reach a certain threshold (approximately 60% or higher, specificed in the "puzzlebox_brainstorms_configration.ini" file) the "Speed" meter will begin to fill and the Radio Controlled Helicopter will take off in "Hover" mode. Once concentration or relaxation levels fall below that threshold the helicopter automatically land.
If using an Emotiv EPOC headset, you can assign arbitrary detections to helicopter functions by linking through the "EmoKey" program to the following keyboard shortcuts:
"Home " or "[ " - Hover
"Page Up " or "] " - Fly Forward
"End " or "\ " - Land
Layman's Explanation :
(Note: this explanation is not 100% accurate but may be a helpful way to visualize and understand what is going on)
Pretend you are standing outside of a baseball stadium while a game is on. You can't see the players on the field (we can't "read your mind") but every once in awhile, you can hear the crowd cheer and shout and you know something exciting is happening. We know that when this happens, it means you are paying attention. You can think of the electrode which rests on the temple as an antenna that is picking up a broadcast of the crowd cheering. When we tune to just the right station on the dial and hear that roar of excitement, we know that you are concentrating.
The human brain is made up of approximately 100 billion neurons which are constantly exchanging and signaling information through chemical processes that produce electricity. When a region of the brain related to a particular function is highly active, small changes in electrical activity can be measured on the surface of the scalp directly over that region.
Just as with the remote control circuit, we measure these levels and changes in electrical activity involts (although the changes are many orders of magnitude smaller). If you were to pull a single AA battery out of the remote control you would find written on the side "1.5v" which indicates the normal charge of that battery to be 1.5 volts . We measure the electrical signals of the brain in millionths of volts (microvolts), using an EEG which in essence is just an extremely sensitive voltmeter.
When we measure the electrical readings of the brain using an EEG headset, we can use mathematics to process the signal. Coupled with knowledge that the electrode has been placed at the cerebral cortex (the frontal lobe of the brain, right under the forehead), along with measurements taken a neutral ground reference (such as on the user's ear, where there are no neurons ) we can make calculations about levels of attention, focus, and relaxation.
In this Instructable we will use this software to fly our helicopter forward, hover, and/or land automatically based on detected levels of concentration and relaxation from an EEG headset such as the NeuroSky MindSet or Emotiv EPOC .
The first step, after loading the application, is to connect to the Puzzlebox Synapse server which was prepared in the previous step. The server can exist on the same computer system or be accessed remotely across a network or the Internet over TCP/IP. Click on the "Control Panel" tab of Puzzlebox Brainstorms and after verifying the Host and Port settings, click "Connect" to begin receiving EEGdetections.
Next click on the "RC Helicopter" tab.
The transmitter chip should have already been extracted and connected to the computer system as described in earlier steps. The USB-to-Serial interface will appear as a COM port under Windows or a /dev/ttyUSB serial device under Linux. By default, when "Concentration" or "Relaxation" levels reach a certain threshold (approximately 60% or higher, specificed in the "puzzlebox_brainstorms_configration.ini" file) the "Speed" meter will begin to fill and the Radio Controlled Helicopter will take off in "Hover" mode. Once concentration or relaxation levels fall below that threshold the helicopter automatically land.
If using an Emotiv EPOC headset, you can assign arbitrary detections to helicopter functions by linking through the "EmoKey" program to the following keyboard shortcuts:
"Home " or "[ " - Hover
"Page Up " or "] " - Fly Forward
"End " or "\ " - Land
Layman's Explanation :
(Note: this explanation is not 100% accurate but may be a helpful way to visualize and understand what is going on)
Pretend you are standing outside of a baseball stadium while a game is on. You can't see the players on the field (we can't "read your mind") but every once in awhile, you can hear the crowd cheer and shout and you know something exciting is happening. We know that when this happens, it means you are paying attention. You can think of the electrode which rests on the temple as an antenna that is picking up a broadcast of the crowd cheering. When we tune to just the right station on the dial and hear that roar of excitement, we know that you are concentrating.
The human brain is made up of approximately 100 billion neurons which are constantly exchanging and signaling information through chemical processes that produce electricity. When a region of the brain related to a particular function is highly active, small changes in electrical activity can be measured on the surface of the scalp directly over that region.
Just as with the remote control circuit, we measure these levels and changes in electrical activity involts (although the changes are many orders of magnitude smaller). If you were to pull a single AA battery out of the remote control you would find written on the side "1.5v" which indicates the normal charge of that battery to be 1.5 volts . We measure the electrical signals of the brain in millionths of volts (microvolts), using an EEG which in essence is just an extremely sensitive voltmeter.
When we measure the electrical readings of the brain using an EEG headset, we can use mathematics to process the signal. Coupled with knowledge that the electrode has been placed at the cerebral cortex (the frontal lobe of the brain, right under the forehead), along with measurements taken a neutral ground reference (such as on the user's ear, where there are no neurons ) we can make calculations about levels of attention, focus, and relaxation.
