c++ - Reading from .oni file results in out of order frames -
my issue duplicate of this one. there no solution there. in hopes find solution , detail specific setup, below shows function used read frames .oni file. if function run type == 2 (i.e. runs # of rgbd images, criteria #), running function in loop should allow user access each image. however, indices color image , depth image not match , out of order. continues until waitforanystream times out following calls img_pipeline::listen(...).
void img_pipeline::listen(int type, int criteria){ int exitnumber; clock_t start = clock(); double elapsedtime; openni::videoframeref frame; int currentimgcount=0; switch (type){ case 0: { exitnumber = -1; break; } case 1: { exitnumber = criteria; break; } case 2: { exitnumber = -1; break; } } (int = 0;i!=exitnumber;i++){ readystream = -1; rc = openni::openni::waitforanystream(streams, 2, &readystream, sample_read_wait_timeout); if (rc != openni::status_ok) { printf("wait failed! (timeout %d ms)\n%s\n", sample_read_wait_timeout, openni::openni::getextendederror()); //break; } switch (readystream) { case 0: { // depth depth.readframe(&frame); break; } case 1: { // color color.readframe(&frame); break; } default: { printf("unexpected stream: %i\n", readystream); continue; } } int height = frame.getheight(); int width = frame.getwidth(); cvcolor.release(); cvx.release(); cvy.release(); cvz.release(); cvcolor = cv::mat(height, width, cv_8uc3); cvx = cv::mat(height, width, cv_32f); cvy = cv::mat(height, width, cv_32f); cvz = cv::mat(height, width, cv_32f); switch (frame.getvideomode().getpixelformat()) { case openni::pixel_format_depth_1_mm: case openni::pixel_format_depth_100_um: { openni::depthpixel* pdepth = (openni::depthpixel*)frame.getdata(); int k =0; (int ri = 0; ri<height; ri++) { (int ci = 0; ci<width; ci++) { float pdepth_val = pdepth[k]; openni::coordinateconverter::convertdepthtoworld(depth, (float)ri, (float)ci, pdepth_val, &cvx.at<float>(ri,ci), &cvy.at<float>(ri,ci), &cvz.at<float>(ri,ci)); k++; } } totalframes[0]++; xyzcaptured = true; printf("frame index: %i \n", frame.getframeindex()); printf("depth captured. \n"); break; } case openni::pixel_format_rgb888: { cvcolor.data = (uchar*)frame.getdata(); totalframes[1]++; colorcaptured = true; printf("frame index: %i \n", frame.getframeindex()); printf("color captured. \n"); break; } default: printf("unknown format \n"); } printf("frame extracted. \n"); if (colorcaptured && xyzcaptured){ if (newbutnotread == true){ imgsmissed++; } else newbutnotread = true; colorcaptured = false; xyzcaptured = false; rgbd_out.clear(); rgbd_out.push_back(cvx); rgbd_out.push_back(cvy); rgbd_out.push_back(cvz); rgbd_out.push_back(cvcolor); currentimgcount++; printf("image overwritten. \n"); } elapsedtime=(clock()-start)/((double)clocks_per_sec); printf("time since listen initiation: %f \n \n", elapsedtime); if (currentimgcount ==criteria && type == 2) return; else if (elapsedtime>(double)criteria && type==0) return; } frame.release(); } here console output example:
frame index: 1 depth captured. frame extracted. time since listen initiation: 0.004846 frame index: 2 depth captured. frame extracted. time since listen initiation: 0.011601 frame index: 1 color captured. frame extracted. image overwritten. time since listen initiation: 0.012640 depth frame count: 3 color frame count: 2 frame index: 54 color captured. frame extracted. time since listen initiation: 0.000067 frame index: 57 depth captured. frame extracted. image overwritten. time since listen initiation: 0.005878 depth frame count: 4 color frame count: 3 frame index: 96 color captured. frame extracted. time since listen initiation: 0.000079 frame index: 99 depth captured. frame extracted. image overwritten. time since listen initiation: 0.003628 depth frame count: 5 color frame count: 4 frame index: 126 color captured. frame extracted. time since listen initiation: 0.000048 frame index: 130 depth captured. frame extracted. image overwritten. time since listen initiation: 0.004782 depth frame count: 6 color frame count: 5 frame index: 152 color captured. frame extracted. time since listen initiation: 0.000065 frame index: 156 depth captured. frame extracted. image overwritten. time since listen initiation: 0.008294 depth frame count: 7 color frame count: 6 frame index: 181 color captured. frame extracted. time since listen initiation: 0.000045 frame index: 185 depth captured. frame extracted. image overwritten. time since listen initiation: 0.004095 depth frame count: 8 color frame count: 7 frame index: 208 color captured. frame extracted. time since listen initiation: 0.000054 frame index: 212 depth captured. frame extracted. image overwritten. time since listen initiation: 0.004242 depth frame count: 9 color frame count: 8 frame index: 236 color captured. frame extracted. time since listen initiation: 0.000092 frame index: 240 depth captured. frame extracted. image overwritten. time since listen initiation: 0.005918 depth frame count: 10 color frame count: 9 frame index: 261 color captured. frame extracted. time since listen initiation: 0.000731 frame index: 262 color captured. frame extracted. time since listen initiation: 0.000877 frame index: 266 depth captured. frame extracted. image overwritten. time since listen initiation: 0.009347 depth frame count: 11 color frame count: 11 frame index: 286 color captured. frame extracted. time since listen initiation: 0.000047 frame index: 290 depth captured. frame extracted. image overwritten. time since listen initiation: 0.006080 depth frame count: 12 color frame count: 12 frame index: 311 color captured. frame extracted. time since listen initiation: 0.000072 frame index: 315 depth captured. frame extracted. image overwritten. time since listen initiation: 0.006453 depth frame count: 13 color frame count: 13 frame index: 337 color captured. frame extracted. time since listen initiation: 0.000062 frame index: 341 depth captured. frame extracted. image overwritten. time since listen initiation: 0.007485 depth frame count: 14 color frame count: 14 frame index: 367 color captured. frame extracted. time since listen initiation: 0.000042 frame index: 371 depth captured. frame extracted. image overwritten. time since listen initiation: 0.003758 depth frame count: 15 color frame count: 15 frame index: 390 color captured. frame extracted. time since listen initiation: 0.000073 frame index: 395 depth captured. frame extracted. image overwritten. time since listen initiation: 0.007917 depth frame count: 16 color frame count: 16 frame index: 416 color captured. frame extracted. time since listen initiation: 0.000105 frame index: 421 depth captured. frame extracted. image overwritten. time since listen initiation: 0.007554 depth frame count: 17 color frame count: 17 frame index: 453 color captured. frame extracted. time since listen initiation: 0.000060 frame index: 458 depth captured. frame extracted. image overwritten. time since listen initiation: 0.006150 depth frame count: 18 color frame count: 18 frame index: 481 color captured. frame extracted. time since listen initiation: 0.000074 frame index: 486 depth captured. frame extracted. image overwritten. time since listen initiation: 0.007169 depth frame count: 19 color frame count: 19 frame index: 517 color captured. frame extracted. time since listen initiation: 0.000045 frame index: 522 depth captured. frame extracted. image overwritten. time since listen initiation: 0.004196 depth frame count: 20 color frame count: 20 frame index: 547 color captured. frame extracted. time since listen initiation: 0.000071 frame index: 552 depth captured. frame extracted. image overwritten. time since listen initiation: 0.007375 depth frame count: 21 color frame count: 21 frame index: 625 color captured. frame extracted. time since listen initiation: 0.000179 frame index: 631 depth captured. frame extracted. image overwritten. time since listen initiation: 0.007922 depth frame count: 22 color frame count: 22 wait failed! (timeout 2000 ms) waitforstreams: timeout reached unexpected stream: -1 wait failed! (timeout 2000 ms) waitforstreams: timeout reached unexpected stream: -1 wait failed! (timeout 2000 ms) waitforstreams: timeout reached unexpected stream: -1 wait failed! (timeout 2000 ms) waitforstreams: timeout reached unexpected stream: -1 wait failed! (timeout 2000 ms) waitforstreams: timeout reached unexpected stream: -1 wait failed! (timeout 2000 ms) waitforstreams: timeout reached unexpected stream: -1 wait failed! (timeout 2000 ms) waitforstreams: timeout reached unexpected stream: -1 and here call img_pipeline::listen(...):
img_pipeline pip_inst; std::string filename = "/home/derek/test data/rgbd/rgbd_s2_r1"; int type = 2; int criteria = 1; std::vector<cv::mat> out; int nummissedimgs; int start; int stop; pip_inst.connect(filename); while (true) { pip_inst.listen(type, criteria); if (pip_inst.isnewimg()){ out = pip_inst.getimage(); cv::imshow("current frame", out.at(3)); char c = cv::waitkey(0); if (c == 'f') { printf("depth frame count: %i \n", pip_inst.getdepthframecount()); printf("color frame count: %i \n", pip_inst.getcolorframecount()); } else { start = pip_inst.getcolorframecount(); break; } cv::destroywindow("current frame"); } } the color images alternating r, g, b tints. sure issue order data in cv::mat however.
even more interesting, call img_pipeline::listen(...) goes through many frames has different index result running img_pipeline::listen(...) multiple times, incrementing through .oni file.
you can control speed of playback oni files using setspeed command. setting speed -1 ensure frames can read manually in sequence oni stream i.e. each time call waitforanystream guaranteed next frame in stream. see "playback speed" here more details.
Comments
Post a Comment