Real Time Video Processing and Object Detection on Android

documentary Time Video touch and inclination undercover work on mechanical manReal Time Video bear on and aim Detection on android SmartphoneAbstract As Smartphone is getting more potent, can do more superior stuffs that previous required a computer. For employing the richly touch power of Smartphone is mobile computer vision, the ability for a device to capture process analyze understanding of calculates. For mobile computer vision, Smartphone must be straightaway and true(a) while. In this study two applications have been developed on android platform using OpenCV and core library called as CamTest with own implemented algorithms. Efficiency of two Android applications have been compared and found that OpenCV performs faster than CamTest. The results of examining the lift out aspiration signal detection algorithm with reverence to efficiency shows that profuse algorithm has the finest blend of speed and object detection performance. Next projected object light scheme using desist algorithm, which uses SVM, BPNN for training and validation of object in real conviction. The application detects the object perfectly with recognition time around 2 ms using SVM and 1 ms using BPNN.KeywordsAndroid Video Processing object detection SVM debauched corner detector BPNNI. INTRODUCTIONAs Smartphone is the perfect combination of personal digital assistant, media player, camera and several(prenominal) opposite stuffs. It has entirely changed the past about mobile phone. In the early daylights of Smartphone application development only mobile company was able to develop. by and by the introduction of Android OS in 2007, Smartphone application development is richly in demand. Android was developed by Google with Linux core kernel and GNU package stuffs. 16.The introduction of Smartphone with camera Real Time motion picture processing becomes very trendy now and having most critical computation tasks. Nearly all Smartphone applications uses a cam era to use mobile computer vision technology 2. Mobile computer vision technologies playing vital role in developing our day to day activities applications 1.This technology having many objectives like object finding, segmenting, location recognition 2.As Smartphone processors such as MediaTek, ARM, NVIDIA Tegra, and Snapdragon are achieving more computation capability covering a fast growth of mobile computer vision applications, like image editing, augmented reality, object recognition. Long processing time due to the high computational difficulty averts mobile computer vision algorithms from being practically used in mobile phone applications. To overcome this problem, researchers and developers have explored the libraries such as OpenGL and OpenCV 2. Application developers will face lot problems as he does not having basic idea to process real time word picture. OpenCV library is the solution which is written in C, C++ language, reduces the complexity for development and rese arch 17 2.Real-Time recognition and detection of objects is complex and favorite area for research in the todays fast growing mobile computer vision technology. Applications like machine vision, visual surveillance robot navigation are the best examples 4.Object detection and recognition inhabit of three steps basically, first is the feature extraction, second classification and third is the recognition of object using machine learning and several other technologies 3.Due to the growth of Scale Invariant Feature Transform (SIFT)10, the object detection method using interconnected filter changed to key point matching establish object detection method 8 10.SIFT is more focusing on invariant key point matching. On the similar concept new algorithms were innate(p) such as the Speeded-Up Feature Transform (SuRF)11,Center Surrounded Extrema (CenSurE)22, Good Features to Track (GFTT)26, Maximally-Stable Extremal Region Extractor (MSER)24, and Oriented Binary plenteous Independent Elemen tary Features (ORB)21, and Features from speed Segment Test (FAST)12 4 6 8.In this paper, real time video processing efficiency was find using OpenCV 17 and CamTest with support of core library. Next analyze best object detection algorithm with respect to efficiency in support with OpenCV library. Projected real time object recognition system using FAST algorithm 12, SVM 15 and BPNN 25. All the stuffs have been conducted on LG Optimus Vu Smartphone with Android 4.0.4 OS.II. ANDROID ARCHITECTUREThe Android operating system is like other Smartphone OS, with stacked structure 216. Android operating system stack consist on several layers such Kernel Layer,System Libraries,Dalvik Virtual Machine layer (i.e Android Runtime layer),Application spewwork layer and on top Applications layer 216.The Kernel gives basic funtionalities like network management memory management, process management, device management. Libraries are used for different oprations like net income security 216.Android Runtime consist of Dalvik Virtual Machine which is optimized for Android and provides core libraries.The Application Framework layer gives operate to the installed applications in the form of Java Class Library. 216.Application developers takes the services of this layer for application development 216.Application layer is the top layer in the stack where your application will get install 216.III. OPENCV IN ANDROIDThe OpenCV library was officially developed and instald by Intel in 1999 to enforce CPU and GPU exhaustive application 17. The earlier version of OpenCV was written in C27. From the edition 2.0 OpenCV provided both C and C++ interfaces27. In the next edition of 2.2 they had introduces Android port with some sample applications of image processing. Currently it has several optimized methods with the version OpenCV 2.4.927 17.IV. real time video processing methodsTo find and compare the efficiency of OpenCV and CamTest, each processing method of mobile computer vision was applied and average value was calculated 2. The stimulant drug format of video frame should be in standard form such as RGB space227.The input video frame to RGB conversion is done by following relation 28R = 1.164(Y 16) + 1.596(V 128)G = 1.164(Y 16) 0.813(V 128) 0.391(U 128)B = 1.164(Y 16) + 2.018(U 128) (1)Each picture element of video frame is threshold with a constant lean T. If it is greater than T, pixel will be set 1, otherwise 0. g(x,y) = 1, if f(x,y) T = 0, otherwise (2)Where f(x, y) is the original frame and g(x, y) is the threshold frame. The descriptions of processing methods are shown in shelve I.TABLE I. put together PRCESSING METHODS AND ITSDESCRIPTIONV. METHODOLOGYFirst designed application layout using JAVA and XML. Then, the processing methods and object detection algorithms are written using JAVA and OpenCV. The tools used for blueprint and programming are Android SDK 16, OpenCV 17 and JAVA SDK.Application file is then installed to the LG Optimus V u. If there are no errors, then started to measure the result regarding frame processing rate. afterward all the data had been collected, and the result is analyzed and compared with the theory. The Application flow is shown in Fig.1.0 and Fig.1.1A) System Flow of Real Time Video Processing and Object Detection AlgorithmsNoYesNextRealTimeVideoFramezFig. 1.0 Real time video processing flowB) System Flow of Real Time Object Detection AlgorithmsNo Yes Next Real Time VideoFrameFig. 1.1 Real time object detection algorithms flow.VI. EXPERIMENT RESULTSA) execution of instrument of Real Time Video Processing MethodsFor the calculation of processing efficiency of OpenCV and CamTest is calculated by following formula. (7)The unit of FPR is frames processed per second i.e. fps. If the value of Frame Processing Rate(FPR) is high for the particular processing metohd then theat method is more efficient. Higher the value of FPR represents the method is more efficient. Table II. Shows real time video processing methods and frames processed per second by CamTest, OpenCV test.TABLE II. REAL TIME VIDEO PROCESSING METHODS AND FPS OF CAMTEST AND OPENCV TESTFrame Processing proportionality is as follows,FPR Ratio = (OpenCV FPR CamTest FPR)/OpenCV FPR (8)As from Table II, FPR shows significant differences between OpenCV and CamTest.If there is Positive FPR ratio value e.g N, then OpenCV is 1/N times better than CamTest.If there is cast out FPR ratio value e.g M,then CamTest is 1/M times better than OpenCV.As shown in Table III, Frame Processing Rate Ratio(average) is 0.64,leads to a conclusion that OpenCV (1/0.64 times) 1.56 times faster and better than CamTest.TABLE III. REAL TIME VIDEO PROCESSING METHODS AND FPR RATIOFig. 2.0 Frame processing rate using CamTest and OpenCV test for eight image processing methods.B) Performance of Real Time Object Detection AlgorithmsTABLE IV. REAL TIME OBJECT DETECTION ALGORITHMSAND THEIR FPSFig. 2.1 Frame Processing Rate for object detection algorithm.As shown in Table IV and Fig. 2.1, FAST algorithm is having the highest fps value and 10 times faster as compare to SIFT and SURF.The minimus fps for real time object recognition should be at least 15 fps and FAST achieves the near same thing. So that FAST is having optimum performance in real time scenario while executing real time object detection operation.VII. APPLICATIONAs from experimental results shown above in Table IV, we concluded that FAST algorithm 12 is almost several times faster than other algorithms. To recognize the object in real time video FAST algorithm almost achieves 15 fps. As FAST algorithm extracts the corner features accurately and it requires less time for it. So proposed a Real Time Object course credit system using FAST algorithm is as follows.A) System Flow of Real Time Object recognitionAs shown in Fig. 3.0 Input object image is captured by Smartphone camera and it is saved to internal storage. FAST corner detector 12 algorithm is applied on the captured image to extract the features. The extracted features should have the same number and location as the viewpoint and corner changes. So the extracted features should be adjusted to the same number and it called as normalization. After the features are adjusted to the same number, weightiness is calculated for SVM 15 and BPNN 25 for training the features. After that feature database will get created. After the preparation of database object will get recognized in real time video via SVM 15 and BPNN 25. As system recognizes the object it shows the feature count and recognition time on the display of Smartphone.No Input Database YesFig. 3.0 Real Time Object Recognition FlowA) ResultsThe Real time object recognition system shown above in Fig. 3.0 was developed for LG Optimus Vu and Android platform 4.0.4. The development environment consist of Microsoft Windows 7 with Intel Core i3,2GB RAM,Android SDK,NDK and JAVA SDK.The object used for training was Hand Watch and trai ning time was 102 ms using SVM and 1115 ms using BPNN.The Table V presents the recognition time for object (Hand Watch) using FAST corner detector, SVM and BPNN.TABLE V. RECOGNITION TIME FOR HAND WATCH OBJECTVIII. CONCLUSIONAs per the above experimentation and results, Most of the real time video processing methods executed using OpenCV having high performance with respect to efficiency than the CamTest. OpenCV gives more watchfulness towards the efficinecy than the CamTest.As per the result obtained from the real time object detction application, FAST algorithm achieves high efficiency, almost 15 fps compared to other algorithms.For the futurescope, like to enhance the FAST algorithm in terms of accuracy.The proposed real time object recognition system gives faster and accurate recognition of object(Hand Watch) on the Smartphone using SVM and BPNN. In future would like to introduce multi object recognition, location tracking on Smartphone platforms,also like to introduce the conce pt like GPU and parallel computing with OpenCL.REFERENCES1 Nasser Kehtarnavaz and Mark Gamadia, Real-Time Image and Video Processing From Research to Reality, Synthesis Lectures On Image, Video and Multimedia Processing Lecture 5, 2006.2 Khairul Muzzammil bin Saipullah and Ammar Anuar, Real-Time Video Processing Using Native scheduling on Android Platform, 8th IEEE International Colloquium on Signal Processing and its Applications, 2012.3 Kanghun Jeong and Hyeonjoon Moon, Object Detection using FAST Corner Detector based on Smartphone Platforms, First ACIS/JNU International Conference on Computers, Networks, Systems, and Industrial applied science, 2011.4 Paul Viola, Michael Jones, Robust Real-time Object Detection, Second International Workshop on Statistical and Computational Theories of survey, July2001.5 L. Zhang and D. Yan, An Improved Morphological Gradient coast Detection Algorithm, IEEE International Symposium on Communications and Information Technology (ISCIT), Vol. 2, pp. 1280-1283, 2005.6 O. Folorunso, O. R. Vincent and B. M. Dansu, Image bunt detection, A knowledge management technique for visual scene analysis, Information Management and Computer Security, Vol. 15, No. 1, pp. 23-32, 2004.7 D. G. Kamdar and C. H. Vithalani, Simulation and Performance Evaluation of Edge Detection Techniques in Differential Time Lapse Video, IEEE International conference on Computational Intelligence and Computing Research (ICCIC), 20128 David G. Lowe, Object Recognition from Local Scale-Invariant Features, the Proceedings of the Seventh IEEE International Conference on Computer Vision, pp. 1150-1157, 1999.9 Clemens Arth, Christian Leistner, Robust Local Features and their Application in Self Calibration and Object Recognition on Embedded Systems, Computer Vision and Pattern Recognition, 2007. IEEE Conference, June 2007.10 D. G. Lowe, Distinctive Image Features from Scale-Invariant Key points, International journal of Computer Vision, 60(2), pp. 91-110, 2004.11 H. Bay, T. Tuytelaars, L. V. Gool, SURF Speeded Up Robust Features, Proceedings of the European Conference on Computer Vision, 2006.12 E. Rosten and T. Drummond, Machine learning for high-speed corner detection, European Conference on Computer Vision, Vol. 1, pp. 430-443, 2006.13 E. Rosten and T. Drummond, Fusing points and lines for high performance tracking, Tenth IEEE International Conference on Computer Vision (ICCV05), Vol. 2, pp.1508-1515, 2005.14 Y. Hirose, K. Yamashita and S. Hijiya, Back-Propagation Algorithm Which Varies the Number of Hidden Units, Neural Networks, Vol. 4, pp.61-66, 1991.15 E. J. A. K. Suykens and J. Vandewalle, Least Squares Support sender Machine Classifiers, Neural Processing Letters (ICCV05), Vol. 9, pp. 293- 300, 1999.16K. Owen, an Executive Summary of Research in Android Integrated Development Environments, April 2011.17OpenCV, Open source Computer Vision library. In http//opencv.willowgarage.com/wiki/, 2009.18 Edward Roston and Tom Drummond, Machin e Learning for high-speed corner detection, Department of Engineering, Cambridge University, UK.19 Y. Khairul Muzzammil bin Saipullah and Ammar Anuar, Coparision of Feature Extractors for Real-time Object Detection on Android Smartphone, Journal of Theoretical and Applied Information Technology, Vol. 47, 2013.20 Y. Khairul Muzzammil bin Saipullah and Ammar Anuar, Analysis of Real-time Object Detection Methods for Android Smartphone, 3rd International Conference on Engineering and ICT (ICEI2012), 2012.21 Ethan Rublee Vincent,Gary Bradski, ORB an efficient alternative to SIFT or SURF22 Agrawal M,K. Konolige,CenSurE Center Surround Extremas for Realtime Feature Detection and Matching, Computer Vision ECCV, springer spaniel Berlin/Heidelberg,vol.5305,pp.102-115,200823 Calonder M,V.Lepetit, BRIEFBinary Robust Independent Elemetary Features, Computer Vision ECCV, Springer Berlin/Heidelberg,vol.6314,pp.778-792,201024 Donoser M,Bischof H, Efficeint Maximally Stable External Region(MSER) Tracking, IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp.533-560,June 200625 Y Hirose,K. Yamashita, Back-Propagation Algorithm which varies the Number of Hidden Units, Neural Networks,vol 4, pp.61-66,199126 Shi J,Tomasi C, Good Features To Track In Computer Vision and Pattern Recognition, CVPR 1994,pages 593-600,199427 Amar Anuar,Khairul Muzzamil Saipullah, OpenCV Based Real-Time Video processing Using Native Android Smartphone, International Journal of Computer Technology and Electronics Engineering(IJCTEE),Vol 1,Issue 328 Khairul Muzzamil Saipullah ,Amar Anuar, Measuring Power inhalation For Image Processing On Android Smartphone , American Journal of Applied Sciences,pages 2052-2057,2012