<<<For JCVKinect>>>
Getting started with Jkinect
While there are a number of open source kinect drivers available, there has been a relatively smaller support for Java developers who wish to interact with kinect sensor from Java. When I started the JKinect project, there were already a number of Java wrappers around existing kinect drivers.
However all of them were hard to install and set up; many people using were complaining from getting errors, such as the very common UnsatisfiedLinkError (caused by problems in accessing native dlls from Java). I created Jkinect to provide a very easy-to-install and error-free kinect Java API for the community. JKinect is a simple JNI wrapper around OpenNI’s kinect driver. In this tutorial, we are going to set up Jkinect and write a small java application that gets RGB and depth frames from Kinect and renders them on the screen.
Driver and Jkinect Installation
JKinect currently supports only Windows (Linux and Mac support is on the roadmap). I have tested JKinect under these settings: Windows 7 (32 and 64-bit), JDK 1.6( 32-bit). If you have other Kinect drivers installed on your machine, first uninstall them .
Then, download JKinect from here and unzip it in an arbitrary location. If you look at jkinect-0.0.1 folder, you will see three sub-folders and one zipped file:
- jkinect: this directory contains source and binaries for JKinect itself (JNI code that calls OpenNI).
- jkinect_native: this directory contains a Visual Studio/C++ project. Native functions are implemented here. These are native peers of JNI methods in the JKinect project.
- jkinect-samples: this contains sample applications using JKinect.
- KinectInstall zipped file: this contains Kinect OpenNI drivers plus NITE.
If you unzip KinectInstall.zip, you will see a KinectInstall application. Run it. This will install the PrimeSense/OpenNI’s Kinect driver plus NITE on your machine. Now, if you look at your device manager (control panel|device manager), you should see kinect cameras there. You should also be able to run sample applications from your start menu (start menu|OpenNI|samples ).
The Simple Viewer application
In this sample application, we want to get raw RGB and depth image streams from Kinect and display them. In order to use Jkinect in a Java project, you need to include jkinect-0.0.1.jar and jkinect_native.dll into your project. You can put jkinect_native.dll at the root directory of your project. Also, you will need to put a configuration file called SampleScene.xml at the project root. In this file, you can set a number of settings such as frame rates and image resolutions (for depth and RGB images). For simplicity,you can always use jkinect-samples as a project template and add your own code there. JKinect provides a simple and easy-to-use API for accessing kinect. You just need to create an instane of the JKinect class and call its init method. After that, each time you want to get depth or RGB frames justmake a call to the update method and then getDepth or getGrayscaleImage or getRGB24Image to get depth or RGB or grayscale images, respectively. Finally call the shutdown method upon exiting the application to terminate your session with kinect.
Below is SimpleViewer Java/Swing application code. In this code snippet, the init method initializes the connect device at application start-up. the stop method shuts down kinect when the application ends. the update method gets RGB and depth frames from kinect. This method is called by a timer every few milliseconds. JKinect’s getDepth method returns a one-dimensional array of 16-bit integers, which are depth values. Kinect depth images are 640 pixels in width 480 pixels in height. Depth values returned by the getDepth method are ordered by rows, i.e. the first 640 values constitute the first row of the depth image, the second 640 values represent the second row and so on. To visualize these depth values as a color image, we use different bits of each depth pixel to create Red, Green , and Blue elements of a color pixel. This is done in convertDepthToColor method. The getRGBImage method returns a one-dimensional array of integers that are three bands of RGB mode. These three bands come in this order: R, G, and then B. That is that R values for all pixels come first, followed by G values, and then B values, and again in each color band values are ordered by rows. For example, if the resolution is set to 640*480, then the first 640 values represent the first row, second 640 values represent the second row and so on. You can find this code in jkinect-samples directory. The sample application in jkinect-samples also shows an edge-detected image of depth frame, which is not covered in this tutorial. To run this application go to jkinect-samples/dist directory from a command prompt and then type:
java –jar jkinect-samples-0.0.1.jar
This sample project comes with an ant build file, you need to have Apache ant installed on your computer in order to be able to rebuild the project.
//these labels are used to show depth and RGB images to users
private JLabel rgbFramesLbl;
private JLabel depthFramesLbl;
private JLabel rgbFramesLbl;
private JLabel depthFramesLbl;
//these BufferedImages hold RGB and Depth images
private BufferedImage rgbImg = null;
private BufferedImage depthImg = null;
private BufferedImage rgbImg = null;
private BufferedImage depthImg = null;
// JKinect clasinstance using which you connect to kinect device
private JKinect jKinect;
public void init(){
jKinect = new JKinect();
System.out.println(”initializing kinect…”);
jKinect.init();
System.out.println(”initializing kinect…”);
jKinect.init();
//here initialize labels and add them to your form
…
// this timer calls actionPerformed method every 100 milliseconds, we update RGB and depth images there
new Timer(100, this).start();
new Timer(100, this).start();
}
public void update() {
// we have to call this function before getting next RGB and depth frames
jKinect.update();
jKinect.update();
//create a new BufferedImage to hold depth image
BufferedImage depthimage = new BufferedImage(640, 480,
BufferedImage.TYPE_INT_RGB);
int[] depthFrame = jKinect.getDepth();
WritableRaster wr = depthimage.getRaster();
BufferedImage depthimage = new BufferedImage(640, 480,
BufferedImage.TYPE_INT_RGB);
int[] depthFrame = jKinect.getDepth();
WritableRaster wr = depthimage.getRaster();
// we use convertDepthToColor method to convert depth q6 bit values to RGB colors
wr.setSamples(0, 0, 640, 480, 1, convertDepthToColor(depthFrame));
wr.setSamples(0, 0, 640, 480, 1, convertDepthToColor(depthFrame));
// resizes image to its half size
BufferedImage resizedDepthImage= new BufferedImage(320, 240,
BufferedImage.TYPE_INT_RGB);
Graphics2D g1 = resizedDepthImage.createGraphics();
g1.drawImage(depthimage, 0, 0, 320, 240, null);
g1.dispose();
depthImg = resizedDepthImage;
BufferedImage resizedDepthImage= new BufferedImage(320, 240,
BufferedImage.TYPE_INT_RGB);
Graphics2D g1 = resizedDepthImage.createGraphics();
g1.drawImage(depthimage, 0, 0, 320, 240, null);
g1.dispose();
depthImg = resizedDepthImage;
byte[] rawRGB = jKinect.getGRGB24Image();
BufferedImage image2 = new BufferedImage(640, 480,
BufferedImage.TYPE_INT_RGB);
for (int c = 0; c < 640; c++)
for (int r = 0; r < 480; r++)
{
BufferedImage image2 = new BufferedImage(640, 480,
BufferedImage.TYPE_INT_RGB);
for (int c = 0; c < 640; c++)
for (int r = 0; r < 480; r++)
{
// get R, G, and B values for each pixel and then create a sigle integer RGB from it
int index = r * 640 + c;
int red = rawRGB[index] & 0xFF;
int green = rawRGB[index + (640 * 480)] & 0xFF;
int blue = rawRGB[index + (2 * 640 * 480)] & 0xFF;
int rgb = (red << 16) | (green << 8) | blue;
int index = r * 640 + c;
int red = rawRGB[index] & 0xFF;
int green = rawRGB[index + (640 * 480)] & 0xFF;
int blue = rawRGB[index + (2 * 640 * 480)] & 0xFF;
int rgb = (red << 16) | (green << 8) | blue;
//set pixel value at c,r to rgb value
image2.setRGB(c, r, rgb);
}
//resize image to half
}
//resize image to half
BufferedImage resizedRgbImage = new BufferedImage(320, 240,
BufferedImage.TYPE_INT_RGB);
Graphics2D g = resizedRgbImage.createGraphics();
g.drawImage(image2, 0, 0, 320, 240, null);
g.dispose();
rgbImg = resizedRgbImage;
BufferedImage.TYPE_INT_RGB);
Graphics2D g = resizedRgbImage.createGraphics();
g.drawImage(image2, 0, 0, 320, 240, null);
g.dispose();
rgbImg = resizedRgbImage;
}
//when repaint method is called in turn itself calls this method
public void paintComponent(Graphics g) {
super.paintComponent(g);
if (rgbImg != null) {
rgbFramesLbl.setIcon(new ImageIcon(rgbImg));
depthFramesLbl.setIcon(new ImageIcon(depthImg));
depthEdgeFramesLbl.setIcon(new ImageIcon(edgeImg));
super.paintComponent(g);
if (rgbImg != null) {
rgbFramesLbl.setIcon(new ImageIcon(rgbImg));
depthFramesLbl.setIcon(new ImageIcon(depthImg));
depthEdgeFramesLbl.setIcon(new ImageIcon(edgeImg));
}
}
//this method is called when application is terminating
public void stop() {
System.out.println(”shutting down…”);
jKinect.shutDown();
}
System.out.println(”shutting down…”);
jKinect.shutDown();
}
// timer object calls this method every 100 milliseconds to get the updated depth and RGB frames
public void actionPerformed(ActionEvent arg0) {
update();
repaint();
}
public void actionPerformed(ActionEvent arg0) {
update();
repaint();
}
//converts 16-bit depth values to RGB color values, this is just a sample view to convert
// depth to color, you can do your own way to
public static int[] convertDepthToColor(int[] depthPixels) {
int colorPixels[] = new int[depthPixels.length];
for (int i = 0; i < depthPixels.length; i++) {
int red = (depthPixels[i] >> 2) & 0xFF;
int green = (depthPixels[i] >> 4) & 0xFF;
int blue = (depthPixels[i] >> 5) & 0xFF;
colorPixels[i] = (red << 16) | // red
(green << 8) | // green
blue; // blue
int colorPixels[] = new int[depthPixels.length];
for (int i = 0; i < depthPixels.length; i++) {
int red = (depthPixels[i] >> 2) & 0xFF;
int green = (depthPixels[i] >> 4) & 0xFF;
int blue = (depthPixels[i] >> 5) & 0xFF;
colorPixels[i] = (red << 16) | // red
(green << 8) | // green
blue; // blue
}
return colorPixels;
}
}
Conclusion
In this tutorial, I demonstrated you how to set up kinect device on your machine and wrote a small Java/swing application that gets depth and RGB frames from kinect and renders them on the screen. Java Advanced Imaging API is also included in the jkinect-samples project, so you can use Java Advanced Imaging API to perform advanced image processing operations.
Keep commenting.....
