sustainability Article Design and Analysis of Multiple OS Implementation on a Single ARM-Based Embedded Platform Byoungwook Kim 1 and Min Choi 2, * 1 2 * Creative Informatics & Computing Institute, Korea University, 145 Anam-ro, Seongbuk-gu, Seoul 02841, Korea; [email protected] Department of Information and Communication Engineering, Chungbuk National University, 1 Chungdae-ro, Seowon-gu, Cheongju, Chungbuk 28644, Korea Correspondence: [email protected]; Tel.: +82-10-9969-5699 Academic Editor: James J. Park Received: 5 April 2017; Accepted: 15 April 2017; Published: 25 April 2017 Abstract: Recently, with the development of embedded system hardware technology, there is a need to support various kinds of operating system (OS) operation in embedded systems. In mobile processors, ARM started to provide the virtualization extension support technology which was intended for processors in PC processors. Virtualization technology has the advantage of using hardware resources effectively. If the real-time operating system (RTOS) is operated on a hypervisor, there is a problem that RTOS performance is degraded due to overhead. Thus, we need to compare the performance between a single execution of the RTOS and simultaneous execution of multiple OS (RTOS + Linux). Therefore, in this paper, we measure the performance when the RTOS operates independently on the NVidia Jetson TK-1 embedded board supporting virtualization technology. Then, we measure the performance when the RTOS and Linux are operating simultaneously on top of a hypervisor. For this purpose, we implemented and ported such a RTOS, especially FreeRTOS and uC/OS, onto two embedded boards, such as the Arndale board (SAMSUNG, Seoul, South Korea) and the NVidia TK1 board (NVIDIA, Santa Clara, CA, USA). Keywords: embedded system; real-time operating system; performance evaluation; multiple operating system 1. Introduction Recently, embedded systems can run multiple operating systems through virtualization technology. Through this hypervisor, we implemented an embedded system that simultaneously runs RTOS and Linux on a single platform. The RTOS performs hardware control tasks requiring a real-time property. At the same time, Linux runs a variety of user programs. Therefore, a highly-reliable high-performance embedded system can be realized. Multiple operating systems on a single platform using a hypervisor provides isolation between the operating systems, but the requirement for an embedded hypervisor are distinct from hypervisors targeting servers and applications [1–3]. While desktop and enterprise environments use hypervisors to consolidate hardware and isolate computing environments from one another, in an embedded system, the various components typically function collectively to provide the device’s functionality [4,5]. An implementation of an embedded hypervisor must deal with a number of issues, which include the highly-integrated nature of embedded systems, the requirement for isolated functional blocks within the system to communicate rapidly, the need for real-time/deterministic performance, the resource-constrained target environment, and the wide range of security and reliability requirements [6]. When a hypervisor does not provide a real-time property, the context switching overhead causes a delay, resulting in performance degradation in the guest operating system. This paper compares and analyzes the performance between Linux and a RTOS Sustainability 2017, 9, 684; doi:10.3390/su9050684 www.mdpi.com/journal/sustainability Sustainability 2017, 9, 684 2 of 14 on a single embedded platform. In order to conduct the experiment, we make use of the Rhealstone benchmark, which was developed by Boger et al. The rest of this paper is organized as follows: Section 2 describes background. Sections 3 and 4 provide system design and experimental results. Finally, we conclude and summarize our work in Section 5. 2. Materials and Methods A hypervisor is a middleware that can run multiple guest operating systems on a single type of hardware. The hypervisor monitors and manages the resources used by the guest OS [7,8]. In this environment, each OS is isolated as if it were running alone. The ARM Cortex-A15 processor’s Virtualization Extension features include: (1) Introduction of Privileged Level HYP Mode In the ARM Virtual Extension, a hypervisor can have higher privileges than the guest OS through the Hyp mode. It is not necessary for the guest OS to recognize the existence of the privileged mode, Hyp mode, which is newly introduced in ARM Cortex-15. The guest OS can still operate in supervisor mode and user mode. On the ARMv7 Cortex-A15 processor, the OS kernel uses supervisor mode. On the ARMv7 Cortex-A15 processor, the user application uses the user mode. In addition, the virtualization hypervisor uses Hyp mode. The hypervisor mode has access to special registers that the user mode or supervisor cannot access. For example, the hypervisor mode has its own Stack Pointer (SP), Saved Processor Status Register (SPSR), and Exception Link Register (ELR). Additionally, registers, such as physical counters, are accessible only in hypervisor mode. (2) Interrupt Virtualization: GICv2 Previously, if an interrupt occurred during the operation of the guest OS, the virtual machine would have to generate a trap to switch to hypervisor mode first and then process the interrupt. On the other hand, the ARM Cortex-A15 processor introduces the concept of virtual interrupt to eliminate this inconvenience as shown in Figure 1. A virtual interrupt allows the virtual machine to receive an interrupt signal directly or to clear an interrupt that has occurred [9,10]. The virtual machine can process its own interrupt at the guest OS level without the hypervisor’s intervention (however, hardware and device drivers must support the virtual CPU interface for this purpose). In the ARM Cortex-A15, the generic interrupt controller can collect/analyze and arbitrate a large number of interrupt sources: (1) interrupt masking; (2) interrupt priority; (3) interrupt distribution for target processor; (4) interrupt status tracking; and (5) interrupt generation for software. In the ARM Cortex-A15, the interrupt can be (1) passed to the currently-running guest OS; (2) passed to the other guest OS; (3) passed to the hypervisor; or (4) passed to the OS or RTOS which runs in the secure TrustZone. In the ARM Cortex-A15, the hypervisor takes priority in physically interrupting. However, if the interrupt is an interrupt to be passed to the guest OS, the hypervisor provides the ability to map the virtual interrupt to the guest OS. In addition, a hypervisor can create a virtual interrupt even if no physical interrupt has occurred [11,12]. Sustainability 2017, 9, 684 3 of 14 Sustainability 2017, 9, 684 3 of 14 Figure1.1.Generic Genericinterrupt interrupt controller Figure controllerarchitecture. architecture. (3) (3) Timer Virtualization Support: GTimer Timer Virtualization Support: GTimer System counter: Gtimer is a shareable “always on” counter. Gtimer has fast read access System counter: Gtimer a shareableIt “always on”count counter. has fast access capability capability for reliable time is distribution. has a 64-bit valueGtimer and counts fromread a value of zero at for reliable time distribution. It system has a 64-bit value and counts fromisa the value of zero at initialization. initialization. The operating readscount the CNTFRQ register, which system control register The at operating system reads CNTFRQ register, whichthe is the system control registerthrough at system system boot time. Thethe operating system initializes system counter frequency theboot CNTFRQ register and uses the system counter. time. The operating system initializes the system counter frequency through the CNTFRQ register and uses the Virtual system counter: counter.Virtual counters allow virtualization to measure time on each virtual machine during operation. All processors generic timers always providetime virtual counters, even if Virtual counter: Virtual counterswith allow virtualization to measure on each virtual machine virtualization extension technologywith is notgeneric applied timers (at this time, the provide virtual counter represents virtual during operation. All processors always virtual counters, even if time). If the virtualization extension technology is not applied, the virtual time is the same as the virtualization extension technology is not applied (at this time, the virtual counter represents virtual physical time. The virtual counter has the same value as the physical counter. If virtualization time). If the virtualization extension technology is not applied, the virtual time is the same as the extension technology is applied, the virtual counter has a value obtained by subtracting the virtual physical time. The virtual counter has the same value as the physical counter. If virtualization extension offset of 64 bits from the physical counter. The CNTVOFF register is an RW register. It is accessible in technology is applied, the virtual counter a value obtained byit subtracting the virtual offset of the non-secure hypervisor mode [13]. If thehas SCR.NS value is set to 1, is accessible even in the secure 64 bits frommode. the physical counter. CNTVOFF anCNTVCT RW register. It is in the monitor The current virtualThe counter value isregister stored inisthe register as accessible 64 bits. If one non-secure mode [13]. If thespecial SCR.NS value is to 1, is accessible in the secure wants tohypervisor access this register, they need privileges to set access theitregister [14,15]even (you can access monitor mode. on The current counter value is stored the CNTVCT register as 64 bits. If one this register secure PL1virtual mode, non-secure PL1 mode, andinnon-secure PL2 mode). that raises event after privileges a certain period of time: generic timer provides a wants to Timer accessfunction this register, they an need special to access the The register [14,15] (you can access function, which varies slightly depending the processor manufacturer. Timers provide the this timer register on secure PL1 mode, non-secure PL1 on mode, and non-secure PL2 mode). ability send signals the system. If the processor is connected the GIC, signaltimer is transferred Timertofunction that to raises an event after a certain period oftotime: Thethe generic provides a through the PPI. When virtualization extension technology (VE) is applied, the processor provides timer function, which varies slightly depending on the processor manufacturer. Timers provide the one non-secure PL1 physical timer, one secure PL1 physical timer, one non-secure secure PL2 ability to send signals to the system. If the processor is connected to the GIC, the signal is transferred physical timer, and one virtual timer. Each timer provides three registers: a 64 bit compare value through the PPI. When virtualization extension technology (VE) is applied, the processor provides register for a 64-bit unsigned up-counter, a 32-bit timer value register for a 32-bit signed one non-secure PL1 physical timer, one secure PL1 physical timer, one non-secure secure PL2 physical down-counter, and a 32-bit control register, which is available as both an up-counter and timer, and one virtual timer. Each timer provides three registers: a 64 bit compare value register for a down-counter. 64-bit unsigned up-counter, a 32-bitsome timer value register a 32-bit signed down-counter, and aon 32-bit The RTOS kernel requires modifications to for support ARM Cortex-A15 virtualization control register, which is available as both an up-counter and down-counter. the hypervisor. To this end, the system structure and source tree of uC/OS-II and FreeRTOS are The RTOS kernel requires some modifications to support ARM Cortex-A15 virtualization on briefly shown. the hypervisor. To this end, the system structure and source tree of uC/OS-II and FreeRTOS are briefly shown. Sustainability 2017, 2017, 9, 6849, 684 Sustainability 4 of 144 of 14 Figure 2 is a2conceptual diagram that changes fromfrom the traditional structure to the which Figure is a conceptual diagram that changes the traditional structure toone the in one in which Figure 2 is a conceptual diagram that changes from the traditional structure to the one in which the the hypervisor is used. The ARMv7 Cortex-A15 processor is implemented by virtualization hypervisor is used. The ARMv7 Cortex-A15 processor is implemented by virtualization the hypervisor is used. The ARMv7 Cortex-A15 processor is implemented by virtualization extensions extensions (VE),(VE), a hardware-supported virtualization to support virtualization technology. It also extensions a hardware-supported virtualization to support virtualization technology. It also (VE), a hardware-supported virtualization to support virtualization technology. It also supports supports interrupt virtualization technology (GICv2). The GIC is a technology that reduces the supports interrupt virtualization technology (GICv2). The GIC is a technology that reduces the interrupt virtualization technology (GICv2). The GIC is without aOS technology that reduces the by overhead by handling the interrupts directly by the guest OS intervention of the hypervisor overhead by handling the interrupts directly by the guest without intervention of theoverhead hypervisor handling the interrupts directly by the guest OS without intervention of the hypervisor when the whenwhen the interrupt occurs. Therefore, the guest OS needs to operate in the mode and the the interrupt occurs. Therefore, the guest OS needs to operate insupervisor the supervisor mode and the interrupt occurs. Therefore, the guest OS needs to operate in the supervisor mode and the user mode user user mode in the same manner as when it is executed by itself. There are various RTOSs in the mode in the same manner as when it is executed by itself. There are various RTOSs in the in the same manner as when it is executed by itself. There are various RTOSs in the market. However, market. However, in this study, uC/OS-II and FreeRTOS are used, for which the source code is free market. However, in this study, uC/OS-II and FreeRTOS are used, for which the source code is free in this study, uC/OS-II and FreeRTOS are used, for which the source code is free [16]. [16]. [16]. Figure 2. System architecture. Figure 2. System architecture. Figure 2. System architecture. 3. Design and Implementation 3. Design Design and Implementation Implementation 3. and In study, this we NVidia’s Jetson development board an Cortex-A15 In this we used NVidia’s Jetson TK-1TK-1 development board withwith an ARMv7 Cortex-A15 In this study, study, we used used NVidia’s Jetson TK-1 development board with an ARMv7 ARMv7 Cortex-A15 processor, as shown in Figure 3. We installed the QPlus Hypervisor as a hypervisor in Jetson TK1.TK1. processor, as shown in Figure 3. We installed the QPlus Hypervisor as a hypervisor in Jetson TK1. processor, as shown in Figure 3. We installed the QPlus Hypervisor as a hypervisor in Jetson Linux (Ubuntu 14.04) and FreeRTOS were used as guest OS. We make use of the embedded Linux (Ubuntu 14.04) and FreeRTOS were used as guest OS. We make use of the embedded hardware Linux (Ubuntu 14.04) and FreeRTOS were used as guest OS. We make use of the embedded hardware debugger Lauterbach Trace 32, is which is debugger. a JTAG debugger. The The debugger enables us line tous by debugger Lauterbach Trace 32, which a JTAG The debugger enables us to debug hardware debugger Lauterbach Trace 32, which is a JTAG debugger. debugger enables to debug line by line on the embedded system, dump a segment of memory, view the content and line on the embedded system, dump a segment of memory, view the content and value of a certain debug line by line on the embedded system, dump a segment of memory, view the content and valuevalue of a certain register, and so on. register, so on.register, of and a certain and so on. (a) (a) (b) (b) Figure 3. Working platform and debugger. Note:Note: (a) TK1 boardboard view from from the top; (b) TK1 TK1 and and Figure 3. Working Working platform and debugger. (a) TK1 Figure 3. platform and debugger. Note: (a) TK1 board view view from the top;the (b)top; TK1(b) and Arndale Arndale board with embedded debugger, the Trace32. Arndale board with embedded the Trace32. board with embedded debugger,debugger, the Trace32. We We evaluated the situation of FreeRTOS operating alonealone and and withwith Linux and and FreeRTOS evaluated the situation of FreeRTOS operating Linux FreeRTOS We evaluated the situation of FreeRTOS operating alone and with Linux and FreeRTOS operating operating together with QPlus Hypervisor. In this study, we measured the task transition delay timetime operating together with QPlus Hypervisor. In this study, we measured the task transition delay together with QPlus Hypervisor. In this study, we measured the task transition delay time and and and interrupt latency of FreeRTOS. When running on the Qplus hypervisor, Linux was executed interrupt latency of FreeRTOS. When running on the Qplus hypervisor, Linux was executed interrupt latency of FreeRTOS. When running on the Qplus hypervisor, Linux was executed with a withwith a background process and no running processes. a background process andother no other running processes. background process and no other running processes. Figure 4 shows howhow the uC/OS porting implementation ontoonto the ARM-A15 evaluation board, Figure 4 shows the uC/OS porting implementation the ARM-A15 evaluation board, Figure 4 shows how the uC/OS porting implementation onto the ARM-A15 evaluation board, especially Arndale and and TK1,TK1, progress. The The OS kernel startsstarts withwith startup.S, the ARM assembly especially Arndale progress. OS kernel startup.S, the ARM assembly especially Arndale and TK1, progress. OS kernel starts startup.S, the ARM assembly start-up start-up code.code. ThenThen it calls the main and The OSInit functions in with the 4. Then, it it start-up it calls the main and OSInit functions in left the top left of topthe of Figure the Figure 4. Then, code. Then it calls the main and OSInit functions in the left top of the Figure 4. Then, it initializes initializes and sets several structures and registers related to generic timer, general interrupt initializes and sets several structures and registers related to generic timer, general interrupt and sets several structures registers related to on. generic timer, general interrupt controller, UART, controller, UART, memory, taskand control blocks, and so controller, UART, memory, task control blocks, and so on. memory, task control blocks, and so on. Sustainability 2017, 9, 684 Sustainability 2017, 9, 684 5 of 14 5 of 14 Sustainability 2017, 9, 684 5 of 14 Sustainability 2017, 9, 684 5 of 14 Figure 4. uC/OS kernel architecture. Figure uC/OS kernel Figure 4.4.uC/OS kernelarchitecture. architecture. We make use use of the co-processor interface, CP15, to configure configurethe thegeneric generic timer. generic We make of the co-processor interface, CP15,architecture. to timer. TheThe generic Figure 4. uC/OS kernel We make use of the co-processor interface, CP15, to configure the generic timer. The generic timer timer uses uses virtual timer control, interrupt number27. 27.We We number to 30 virtual timer control, interruptsource sourcePPI4, PPI4, IRQ IRQ number setset thethe IRQIRQ number to 30timer uses virtual timer control, interrupt source PPI4, IRQ number 27. We set the IRQ number to 30 for the Wephysical make use of the co-processor interface, CP15,intoFigure configure the generic The generic for theforphysical timer. The following register settings Figure are required for GTimer timer the timer. The following register settings 55are required fortimer. thethe GTimer timer physical timer. The following register settings in Figure 5the are required forand the GTimer timer interrupt timer virtual timer as control, interrupt source PPI4, IRQ number 27. We set the IRQ number to 30 interrupt be recognized a GIC register. First, we set ICENABLER ISENABLER registers interrupt touses betorecognized aas GIC register. First, we the ICENABLER and ISENABLER registers for the physical timer. The following register settings in Figure 5 are required for the GTimer timer for GICD's interrupt clear and set. Then, we set the ITARGETSR register to set the CPU to recognize to be recognized as a GIC register. First, we set the ICENABLER and ISENABLER registers for GICD's for GICD's interrupt clear and set. Then, we set the ITARGETSR register to set the CPU to recognize interrupt to be recognized as a GIC register. First, we set the ICENABLER and ISENABLER registers the interrupt. In addition, we initialize the timer by setting the bits of the offset corresponding to IRQ interrupt clearInand set. Then, we set thethe ITARGETSR register set of thethe CPU to recognize the interrupt. the interrupt. addition, we initialize timer by setting thetobits offset corresponding to IRQ for GICD's interrupt clear set. Then, we the set the ITARGETSR register to set the CPU to to recognize 27 in thewe IPRIORITYR register to determine priority of the interrupt [17–19]. In in addition, initialize theand timer by setting bits of corresponding IRQ 27 in the 27 the IPRIORITYR register to determine the the priority of the the offset interrupt [17–19]. the interrupt. In addition, we initialize the timer by setting the bits of the offset corresponding to IRQ IPRIORITYR register to determine the priority of the interrupt [17–19]. 27 in the IPRIORITYR register to determine the priority of the interrupt [17–19]. Figure 5. Interrupt processing on hardware initialization. On power-up, or after reset, a processing GIC implementation that supports interrupt grouping is Figure 5. on hardware initialization. Figure 5. Interrupt processing initialization. Figure 5.aInterrupt Interrupt processing onhardware hardware initialization. configured with all interrupts assigned to Group 0, and the FIQ exception request disabled. This Onthat power-up, after a reset, aGIC GIC implementation implementation that interrupt is means Group interrupts are a signaled using the IRQ interrupt request. Figure 6grouping shows this On power-up, or 0or after a reset, thatsupports supports interrupt grouping is On power-up, or interrupts after a reset, a GIC implementation that supports interrupt grouping is configured all assigned Group 0, and request disabled. This This configuration. configured withwith all interrupts assigned totoGroup 0, and the theFIQ FIQexception exception request disabled. configured with all interrupts assigned to Group 0, and the FIQ exception request disabled. This means means Group 0 interrupts signaled using using the the IRQ Figure 6 shows this this means that that Group 0 interrupts arearesignaled IRQinterrupt interruptrequest. request. Figure 6 shows that Group 0 interrupts are signaled using the IRQ interrupt request. Figure 6 shows this configuration. configuration. configuration. Figure 6. Co-processor control register for the use of generic timer. Figure 6. Co-processor control register for the use of generic timer. Figure6. 6.Co-processor Co-processorcontrol controlregister register for for the the use use of of generic generic timer. timer. Figure Sustainability 2017, 9, 684 6 of 14 Sustainability Sustainability2017, 2017,9,9,684 684 6 6ofof1414 When the timer is initialized, the timer periodically generates an interrupt. At this time, uC/OS-II When the timer the timer generates an interrupt. this When the timer isis is initialized, initialized,The theOS timer periodically generates anGIC interrupt. Atdetermine this time, time, checks which interrupt generated. can periodically use the information in the registerAt to uC/OS-II checks interrupt generated. OS use information the GIC toto uC/OS-II checkswhich which interrupt generated. The OScan can usethe the information thenumber. GICregister register the IRQ number. The OS executesisis the interruptThe service routine according to theinin IRQ In the determine the IRQ number. The OS executes the interrupt service routine according to the IRQ determine the IRQ number. The OS executes the interrupt service routine according to the IRQ event of a timer interrupt, the Timer Tick function is called. This function calls the uC/OS-II OS kernel number. InInthe number.OSTimeTick(). theevent eventofofaatimer timerinterrupt, interrupt,the theTimer TimerTick Tickfunction functionisiscalled. called.This Thisfunction functioncalls callsthe the function uC/OS-II OS kernel function OSTimeTick(). uC/OS-II OS kernel function OSTimeTick(). The GIC maintains a state machine for each supported interrupt on each CPU interface. Figure 7 The machine for each supported on The GICmaintains maintains astate state machine forthe each supported interrupt oneach eachCPU CPUinterface. interface.Figure Figure shows anGIC instance of thisastate machine and possible stateinterrupt transitions. 77shows showsan aninstance instanceofofthis thisstate statemachine machineand andthe thepossible possiblestate statetransitions. transitions. Figure 7. State transition diagram for interrupt processing. Figure Figure7.7.State Statetransition transitiondiagram diagramfor forinterrupt interruptprocessing. processing. The study builds on the board totorun The ARM A15 embedded run two orormore more operating systems Thestudy studybuilds buildson onthe theARM ARMA15 A15embedded embeddedboard boardto runtwo twoor moreoperating operatingsystems systems simultaneously through the hypervisor. In particular, we made Ubuntu Linux and RTOS (uC/OS simultaneously simultaneouslythrough throughthe thehypervisor. hypervisor.InInparticular, particular,we wemade madeUbuntu UbuntuLinux Linuxand andRTOS RTOS(uC/OS (uC/OS and FreeRTOS) on the embedded board. Due to the nature of the hypervisor, and run simultaneously andFreeRTOS) FreeRTOS)run runsimultaneously simultaneouslyon onthe theembedded embeddedboard. board.Due Dueto tothe thenature natureof ofthe thehypervisor, hypervisor, two operating systems share the Therefore, as shown in Figure 8,8,the the RTOS two same UART console. twooperating operatingsystems systemsshare sharethe thesame sameUART UARTconsole. console.Therefore, Therefore,as asshown shownin inFigure Figure8, theRTOS RTOS outputs ‘hello arndale’ message and ‘Goodbye arndale’ message to the screen. At the same time, outputs outputsaaa‘hello ‘helloarndale’ arndale’message messageand andaaa‘Goodbye ‘Goodbyearndale’ arndale’message messageto tothe thescreen. screen.At Atthe thesame sametime, time, on Ubuntu Linux, the root user’s Linux prompt is displayed on the screen. on onUbuntu UbuntuLinux, Linux,the theroot rootuser’s user’sLinux Linuxprompt promptisisdisplayed displayedon onthe thescreen. screen. Figure Figure8.8.Simultaneous Simultaneousexecution executionofofmultiple multipleOS OSon onembedded embeddedsystem. system. Simultaneous This study was performed with aaTektronix oscilloscope using the Thisstudy studywas wasperformed performed with Tektronix oscilloscope using theGPIO GPIO ports theJetson Jetson This with a Tektronix oscilloscope using the the GPIO portsports of theofof Jetson TK-1 TK-1 development board to verify the operating characteristics of the hypervisor and guest OS. We TK-1 development board to verify the operating characteristics of the hypervisor and guest OS. We development board to verify the operating characteristics of the hypervisor and guest OS. We also also ofof the benchmark sets [8] comparisons. The also performed performed the Rhealstone Rhealstone benchmark sets [8] for for performance performance comparisons. The performed some some ofsome the Rhealstone benchmark sets [8] for performance comparisons. The Rhealstone Rhealstone benchmark was proposed in 1989. It does not measure the quality of the complete Rhealstone benchmark was proposed in 1989. It does not measure the quality of the complete benchmark was proposed in 1989. It does not measure the quality of the complete solution, but is, solution, aa measurement targeted aa multitasking task solution,a but but is, is, instead, instead, measurement targeted toward toward multitasking solution. In In the thetime task instead, measurement targeted toward a multitasking solution. In the task solution. transition delay transition delay time measurement, we repeatedly output HIGH-LOW to each GPIO port for two transition delay time measurement, we repeatedly output HIGH-LOW to each GPIO port for two measurement, we repeatedly output HIGH-LOW to each GPIO port for two tasks. Each task outputs tasks. Each task outputs high to its GPIO port when it is running. Therefore, after task switching, the tasks. Each task outputs high to its GPIO port when it is running. Therefore, after task switching, the high to its GPIO port when it is running. Therefore, after task switching, the corresponding GPIO port corresponding GPIO port remains low. Task 1 and Task 2 use their own GPIO ports. Therefore, corresponding low. Task 1 and Task 2 use their own GPIOthe ports. Therefore, remains low. TaskGPIO 1 andport Taskremains 2 use their own GPIO ports. Therefore, Figure 9 shows status change Figure 9 shows the status change of each GPIO port simultaneously through the two-channel input Figure 9 shows the status change of each GPIO port simultaneously through the two-channel input of each GPIO port simultaneously through the two-channel input to the oscilloscope. totothe theoscilloscope. oscilloscope. Sustainability 2017, 9, 684 7 of 14 Sustainability 2017, 9, 684 7 of 14 Sustainability 2017, 9, 684 7 of 14 Figure 9. Task switching time. Figure 9. Task switching Figure 9. Task switching time. time. Task switching time is the average time the system takes to switch between two independent Tasksuspended switching time time issleeping, the average average timeof theequal systempriority. takes to to switch switch between two twoisindependent independent and active, not or is tasks Task between switching synchronous and Task switching the time the system takes and active, active,and not is suspended or sleeping, sleeping, tasks of of equal equal priority. Task switching is synchronous synchronous and and not suspended or tasks of priority. Task switching non-preemptive an important measure any multitasking system. is This metric isand influenced non-preemptive and andisisan animportant importantmeasure measureofofany any multitasking system. This metric is influenced non-preemptive multitasking system. This metric is influenced by by the host CPU's architecture, instruction set, and features, and is designed to assess the by the host CPU's architecture, instruction set, and features, and is designed to assess the the host CPU's architecture, instruction set, and features, and is designed to assess the compactness of compactness of taskofcontrol datadata structures and the efficiency efficiency with which the executive manipulates compactness task control with which the executive manipulates task control data structures and thestructures efficiencyand withthe which the executive manipulates the data structures the data in structures saving and restoring contexts. Taskadditionally, switching time, additionally, measures the the data structures in saving and restoring contexts.time, Task switching time, additionally, measures the saving andinrestoring contexts. Task switching measures the executive’s list executive's list management capabilities. Figure 10 shows the related data structures maintained by executive's list management capabilities. Figure 10 shows related data structures by management capabilities. Figure 10 shows the related data the structures maintained by the OSmaintained kernel the OS kernel during the task switching. during the task switching. the OS kernel during the task switching. Figure 10. Related data structures for switching tasks. The task-Switching benchmarking sets up two tasks with equal priority. The tasks switch back and forth between the processor. To first determine the time it takes to perform the for loop “work”, the benchmark just measures theRelated loops performing no work and not task tasks. switching. Figure 10. data for switching Figure 10. Related datastructures structures for switching tasks. As shown in Figure 11, we measured the interrupt latency in the following way. Two tasks output HIGH (Task 1) and LOW (Task 2) status for one GPIO port, respectively. We start the The task-Switching benchmarking sets up taskswith with equal priority. Theswitch tasks back switch back The task-Switching benchmarking uptwo two tasks priority. The tasks measurement when an interrupt occurs.sets It outputs a signal to equal a specific GPIO when the execution forth between the processor. firstdetermine determine the time it an takes to perform the is forfrom loop “work”, and forthand between theinterrupt processor. To To first the time it takes to handler perform the for loop flow enters the handler. Thus, we can check whether interrupt Task 1 or “work”, the benchmark just measures the loops performing no work and not task switching. the benchmark performing no work and not task switching. Task 2 byjust the measures waveform ofthe theloops oscilloscope. As shown in Figure 11, we measured the interrupt latency in the following way. Two tasks output As shown in Figure 11, we measured the interrupt latency in the following way. Two tasks HIGH (Task 1) and LOW (Task 2) status for one GPIO port, respectively. We start the measurement output HIGH 1) occurs. and LOW (Task 2) status for one port, respectively. We when an(Task interrupt It outputs a signal to a specific GPIOGPIO when the execution flow enters thestart the measurement when an interrupt occurs. It outputs a signalhandler to a specific GPIO thethe execution interrupt handler. Thus, we can check whether an interrupt is from Task 1 orwhen Task 2 by waveform of the oscilloscope. flow enters the interrupt handler. Thus, we can check whether an interrupt handler is from Task 1 or Task 2 by the waveform of the oscilloscope. Figure 11. Interrupt processing delay. the benchmark just measures the loops performing no work and not task switching. As shown in Figure 11, we measured the interrupt latency in the following way. Two tasks output HIGH (Task 1) and LOW (Task 2) status for one GPIO port, respectively. We start the measurement when an interrupt occurs. It outputs a signal to a specific GPIO when the execution Sustainability 684 14 flow enters2017, the 9,interrupt handler. Thus, we can check whether an interrupt handler is from Task8 of 1 or Task 2 by the waveform of the oscilloscope. Sustainability 2017, 9, 684 8 of 14 Figure 11. Interrupt processing delay. Figure 11. Interrupt processing delay. 4. Experimental Results 4. Experimental Results Sustainability 2017, 9, 684 8 of 14 In this study, we implemented two real-time operating systems that operate through an ARM In this study, we implemented two real-time operating systems that operate through an ARM A-154. processor with virtualization support: uC/OS and FreeRTOS. We run two ported RTOSs with Experimental A-15 processor Results with virtualization support: uC/OS and FreeRTOS. We run two ported RTOSs with each each Linux operating system. RTOSsgenerate generate a digital in the a wave every Linux operating system. These These two RTOSs a digital clock clock in the of aform waveof every In this study, we implemented real-time operating systems thatform operate through an second. ARM second. We use the generated clock signal to output to the digital timer board, as shown in Figure 12. We use the generated clock signal to output to the digital timer board, as shown in Figure 12. A-15 processor with virtualization support: uC/OS and FreeRTOS. We run two ported RTOSs with each Linux operating system. These RTOSs generate a digital clock in the form of a wave every second. We use the generated clock signal to output to the digital timer board, as shown in Figure 12. Figure 12.12.Digital schematic. Figure Digitaltimer timer board board schematic. Figure 12. Digital timer board schematic. Figure 13 represents simulation results generated by tools SPICE tools Figure 13 representsthe the waveform waveform simulation results generated by SPICE using the using above the Figure 13 represents the waveform simulation results generated by SPICE tools using abovedigital digital timer board schematic of Figure 12.The digital timer board was implemented using timer board schematic of Figure 12. The digital timer board was implemented using digitalthe ICs above digital timer board schematic of Figure 12.The digital timer board was implemented using (7447, Weetc.). connected the digital the clockdigital outputclock from output the evaluation board to two timer boards digital ICs 7490, (7447,etc.). 7490, We connected from the evaluation board to two digital ICs (7447, 7490, etc.). We connected the digital clock output from the evaluation board to two as shown in Figure 14. timer boards as shown in Figure 14. timer boards as shown in Figure 14. Figure 13.Digital Digital timer circuitsimulation. simulation. Figure 13. Digital timer circuit Figure 13. timer circuit simulation. Figure 14 shows the environmental setup for this experiment. We make use of an oscilloscope to measure the digital high/low states when multiple OS and hypervisors are working on the evaluation board. Figure 14. Digital timer circuit simulation environment. Sustainability 2017, 9, 684 9 of 14 Figure 13. Digital timer circuit simulation. Digital timer circuit simulation environment. Figure 14. Digital Figure 14 shows thethe environmental setup for this experiment. We makeboard. use ofWe ancommunicate oscilloscope The server provides operating system image in Tftp on the evaluation to the program digital high/low whenboard multiple OSUART. and hypervisors are board working on thea viameasure the Minicom with the states evaluation and the The evaluation provides evaluation board. clock with a digital timer board. At this time, we measure the clock timing with an oscilloscope to The the server the system We measure clockprovides waveform, as operating shown in Figure 15.image in Tftp on the evaluation board.9 of Sustainability 2017, 9, 684 14 communicate via the Minicom program with the evaluation board and the UART. The evaluation board provides a clock with a digital timer board. At this time, we measure the clock timing with an oscilloscope to measure the clock waveform, as shown in Figure 15. Figure 15. Digital timer circuit simulation. In Figure 15, In Figure 15, the the left left digital digital timer timer board board and and the the right right digital digital timer timer board board may may look look different. different. However, both boards are only show a difference between implementing the same circuit However, both boards are only show a difference between implementing the same circuit on on aa breadboard board. TheThe left left board in Figure 15 receives the clock breadboard and and implementing implementingititonona universal a universal board. board in Figure 15 receives the from the Linux operating system. The right board in Figure 15 receives the clock from the clock from the Linux operating system. The right board in Figure 15 receives the clock from the RTOS RTOS operating system. operating system. Especially, we obtained obtained the the following following four four results results from from this this experiment. experiment. Task switching latency latency Especially, we Task switching measurement results are as follows: Figures 16 and 17 show the first experimental results, which are measurement results are as follows: Figures 16 and 17 show the first experimental results, which are captured from the the oscilloscope. oscilloscope. Of Of the the two two waveforms waveforms shown shown in in the the figures, figures, the the upper upper waveform waveform captured from corresponds to Task 1, and the lower waveform corresponds to Task 2. We set the time axis to 5.00 corresponds to Task 1, and the lower waveform corresponds to Task 2. We set the time axis to 5.00 μs µs when FreeRTOS operates alone. Since one scale on the oscilloscope is 1.00 μs, the task transition when FreeRTOS operates alone. Since one scale on the oscilloscope is 1.00 µs, the task transition delay delay time was measured to be approximately 3.50 μs on average. time was measured to be approximately 3.50 µs on average. Especially, we obtained the following four results from this experiment. Task switching latency measurement results are as follows: Figures 16 and 17 show the first experimental results, which are captured from the oscilloscope. Of the two waveforms shown in the figures, the upper waveform corresponds to Task 1, and the lower waveform corresponds to Task 2. We set the time axis to 5.00 μs when FreeRTOS operates alone. Since one scale on the oscilloscope is 1.00 μs, the task transition Sustainability 2017, 9, 684 10 of 14 delay time was measured to be approximately 3.50 μs on average. Sustainability 2017, 9, 684 Figure Figure 16. 16. Task Task switching time on single RTOS execution. Sustainability 2017, 9, 684 10 of 14 10 of 14 Figure 17. Task switching time on multiple OS (RTOS + Linux) execution. Figure 17. Task switching time on multiple OS (RTOS + Linux) execution. Figure 17. Task switching time on multiple OS (RTOS + Linux) execution. When Linux and FreeRTOS work together through QPlus Hypervisor, the result is as follows. When and workone together through QPlus Hypervisor, thethe result as follows. The time axis Linux was set to FreeRTOS 5.00 ms. Since scale on an oscilloscope is 1.00 ms, taskistransition When Linux and FreeRTOS work together through QPlus Hypervisor, the result is as follows. The time time was axis was set to 5.00 Since onems scale on an oscilloscope 1.00 processes, ms, the taskwhich transition delay measured to bems. about 4.00 on average. There areisthree have delay all The time axis was set to 5.00 ms. Since one scale on an oscilloscope is 1.00 ms, the task transition time was measured to be about 4.00 ms on average. There are three processes, which have all the the same priorities. However, the process “T” starts with the highest priority at initialization time. delay time was measured to be about 4.00 ms on average. There are three processes, which have all same priorities. However, thewill process “T”the starts withby theadding highest1 priority at initialization time. After After initialization, the process reduce priority to the tskIDLE_PRIORITY value the same priorities. However, the process “T” starts with the highest priority at initialization time. initialization, the process will reduce the priority by adding 1 to the tskIDLE_PRIORITY value of itself of itself in order to yield the CPU to the process “F” and process “S”. Figure 18 shows how we After initialization, the process will reduce the priority by adding 1 to the tskIDLE_PRIORITY value in order to yield the CPU to the process “F” and process “S”. Figure 18 shows how we controlled controlled the GPIO ports in the FreeRTOS application tasks, which are used to evaluate the taskthe of itself in order to yield the CPU to the process “F” and process “S”. Figure 18 shows how we GPIO ports the interrupt FreeRTOSdelay. application tasks, which are used to evaluate the task switching time and switching timeinand controlled the GPIO ports in the FreeRTOS application tasks, which are used to evaluate the task interrupt delay. switching time and interrupt delay. Figure 18. GPIO control code example. Figure 18. GPIO control control code code example. example. Figure 18. GPIO The taskYIELD()function yields the CPU to other processes, which is in the ready queue. This code does not actually create tasks, but simply determines the execution time of the portions of the The taskYIELD()function yields the CPU to other processes, which is in the ready queue. This code that are not part of the task-switching measurement. The execution of this code segment is code does not actually create tasks, but simply determines the execution time of the portions of the recorded with the oscilloscope and the second portion of the code runs. The second portion utilizes code that are not part of the task-switching measurement. The execution of this code segment is two tasks. This time the two tasks perform task switching for the desired number of iterations. recorded with the oscilloscope and the second portion of the code runs. The second portion utilizes The result of measuring the interrupt latency is as follows: Among the two waveforms shown in two tasks. This time the two tasks perform task switching for the desired number of iterations. Sustainability 2017, 9, 684 11 of 14 The taskYIELD()function yields the CPU to other processes, which is in the ready queue. This code does not actually create tasks, but simply determines the execution time of the portions of the code that are not part of the task-switching measurement. The execution of this code segment is recorded with the oscilloscope and the second portion of the code runs. The second portion utilizes two tasks. This time the two tasks perform task switching for the desired number of iterations. The result of measuring the interrupt latency is as follows: Among the two waveforms shown in the figure, the upper waveforms are alternating between Task 1 and Task 2 and output HIGH and Sustainability 2017, 9, 684 11 of 14 LOW states. The waveform below, in Figures 19 and 20, are generated from the waveform for the interrupt handler. We calculated the time from when the bottom waveform changes from HIGH to LOW, and when the top waveform changes from LOW to HIGH (or HIGH to LOW). When operating using FreeRTOS alone, we set the time axis to 2.50 µs. Since the interval is 0.50 µs, the interrupt latency is measured to be approximately 3.00 µs on average. Sustainability 2017, 9, 684 11 of 14 Figure 19. Interrupt latency on a single RTOS execution. When Linux and FreeRTOS work together through the QPlus Hypervisor, we set the time axis of the oscilloscope to 250 μs. Since the scale on an oscilloscope is 50 μs, the interrupt latency was measured to be approximately 150 μs on average. Based on the measured results, FreeRTOS operating with Linux through the QPlus Hypervisor showed thatexecution. the task switching delay time is Figure RTOS execution. Figure 19. 19. Interrupt Interrupt latency latency on on aa single single RTOS about 1140 times and the interrupt latency is about 50 times slower than when it is performed alone. When Linux and FreeRTOS work together through the QPlus Hypervisor, we set the time axis of the oscilloscope to 250 μs. Since the scale on an oscilloscope is 50 μs, the interrupt latency was measured to be approximately 150 μs on average. Based on the measured results, FreeRTOS operating with Linux through the QPlus Hypervisor showed that the task switching delay time is about 1140 times and the interrupt latency is about 50 times slower than when it is performed alone. Figure 20. Interrupt Interrupt latency on multiple OS (RTOS + Linux) execution. Based on theand experimental results, we can deduce the following. There is the no time significant When Linux FreeRTOS work together through the QPlus Hypervisor, we set axis of difference in the task transition delay time and interrupt latency time when FreeRTOS operates the oscilloscope to 250 µs. Since the scale on an oscilloscope is 50 µs, the interrupt latency was measured alone. However, the FreeRTOS on theon QPlus Hypervisor showsFreeRTOS a large difference in with task to be approximately 150 µs on operating average. Based the measured results, operating transition delay time. Additionally, the interrupt latency is slowed down. Figure 20. Interrupt latency on multiple OS (RTOS + Linux) execution. Linux through the QPlus Hypervisor showed that the task switching delay time is about 1140 times Figure 21 show our experimental results in which thereitare two digitalalone. timer boards and they and the interrupt latency is about 50 times slower than when is performed Based on the experimental results, we can deduce the following. There is no are supplied clock signal fromresults, two different boards, respectively. They startsignificant counting Based ona the experimental we can digital deducetimer the following. There is no significant difference difference in the task transition delay time and interrupt latency time when FreeRTOS operates the digital timers at the same time, but the timings are skewed occasionally due to the operating in the task transition delay time and interrupt latency time when FreeRTOS operates alone. However, alone. However, the FreeRTOS operating on the QPlus Hypervisor shows a large difference in task system delays [19]. transition delay time. Additionally, the interrupt latency is slowed down. Figure 21 show our experimental results in which there are two digital timer boards and they are supplied a clock signal from two different digital timer boards, respectively. They start counting the digital timers at the same time, but the timings are skewed occasionally due to the operating Sustainability 2017, 9, 684 12 of 14 the FreeRTOS operating on the QPlus Hypervisor shows a large difference in task transition delay time. Additionally, the interrupt latency is slowed down. Figure 21 show our experimental results in which there are two digital timer boards and they are supplied a clock signal from two different digital timer boards, respectively. They start counting the digital timers at the same time, but the timings are skewed occasionally due to the operating system delays [19].2017, 9, 684 Sustainability 12 of 14 (a) (b) (c) (d) Figure 21. Digital timer board execution by clock generated from RTOS and Linux, respectively. Figure 21. Digital timer board execution by clock generated from RTOS and Linux, respectively. Note: (a) at time 00:17; (b) at time 00:55; (c) at time 02:27; (d) at time 03:10 Note: (a) at time 00:17; (b) at time 00:55; (c) at time 02:27; (d) at time 03:10. Importantly, the Cortex-A15 processor supports interrupt virtualization (GICv2). This reduces Importantly, interrupt the Cortex-A15 processor supports interrupt virtualization (GICv2). This reduces the hypervisor’s processing overhead by virtualizing interrupts that occur in each guest the hypervisor’s interrupt processing overhead by virtualizing interrupts that occur in each guest OS. Due to these advantages, it can be seen that the interrupt latency is measured more quicklyOS. in Due theseoperating advantages, it can be seen that the interrupt latency measured more shown quickly in the guest the to guest system and FreeRTOS operation than isthe hypervisor above. The operating andexist FreeRTOS operation the hypervisor shown above. The followingaproblems followingsystem problems when guest OSs than run through hypervisor that do not guarantee real-time exist when guest OSs run through hypervisor that do not guarantee a real-time property. Due to the property. Due to the scheduling of the hypervisor, the RTOS does not match the time actually scheduling of the RTOSTherefore, does not match the time actually operated and thedegradation, time on the operated and the hypervisor, time on the the system. the system causes a slight performance system. Therefore, the system causes a slight performance degradation, as shown in Table 1. as shown in Table 1. Table 1. Experimental results. Table 1. Experimental results. Single RTOS(FreeRTOS) Single RTOS(FreeRTOS) Execution Execution Task switching delay Task switching delay Interrupt delay Interrupt delay 3.5 µs 3.5 μs 3.0 µs 3.0 μs Linux FreeRTOS Linux andand FreeRTOS Simultaneously Simultaneously 4.0 ms 4.0 ms 150 µs 150 μs 5. Conclusions There is a problem that RTOS performance is degraded due to overhead. Thus, we need to compare the performance between a single execution of the RTOS and a simultaneous execution of multiple OSs (RTOS + Linux). Therefore, in this paper, we measured the performance when the Sustainability 2017, 9, 684 13 of 14 5. Conclusions There is a problem that RTOS performance is degraded due to overhead. Thus, we need to compare the performance between a single execution of the RTOS and a simultaneous execution of multiple OSs (RTOS + Linux). Therefore, in this paper, we measured the performance when the RTOS operates independently on the NVidia Jetson TK-1 embedded board supporting virtualization technology. Then, we measured the performance when RTOS and Linux are operated simultaneously on top of a hypervisor. To this end, we implemented and ported such a RTOS, especially FreeRTOS and uC/OS, onto two embedded boards, such as an Arndale board and NVidia TK1 board. There are such problems when multiple guest OSs are running on hypervisor that do not guarantee a real-time property. Due to the scheduling of the hypervisor, the RTOS does not match the time actually operated and the time on the system. Therefore, the system causes a slight performance degradation in a different order of measurement time. Acknowledgments: This work was jointly supported by a Project no. K-17-L01-C01 of Korea Institute of Science and Technology Information, and supported by a Basic Researcher Project No. NRF-2016R1C1B1012189 of National Research Foundation. This work was also jointly supported by a Startup Growth Technology Development Project of Small and Medium Business Administration in Korea. Author Contributions: For the research reported, ByoungWook Kim conceived of and designed the overall architecture and conducted the experiment. Min Choi is the corresponding author, and performed the overall design and the source code implementation for uC/OS II and FreeRTOS porting. Conflicts of Interest: The authors declare no conflict of interest. References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Kihong, L.; Dongwoo, L.; Young, I.E. A Study on Trend for Hardware-assisted Virtualization Technology. In Proceedings of the Korean Information Science Society, Jeju, Korea, 15–16 November 2013; pp. 1316–1318. Young, G.J.; Byung, J.L.; Hee, Y.Y. The comparative analysis of Hypervisor according to Virtualization Method. In Proceedings of the Korean Society of Computer Information Conference, Cheongju, Korea, 22–24 January 2015; pp. 251–253. Travis, L. Exploring the Design of the Cortex-A15 Processor. Available online: https://www.arm.com/files/ pdf/AT-Exploring_the_Design_of_the_Cortex-A15.pdf (accessed on 15 April 2017). In-Kyu, H. K-Hypervisor: Design and Implementation of Hypervisor Based ARM Cortex-A15. Available online: http://www.dbpia.co.kr/Journal/ArticleDetail/NODE02444451?TotalCount=0&Seq=4& isIdentifyAuthor=1&Collection=0&isFullText=0&specificParam=0&SearchMethod=0&Page=1&PageSize= 20 (accessed on 15 April 2017). Prashant, V.; Gernot, H. Hardware-supported virtualization on ARM. In Proceedings of the Second Asia-Pacific Workshop on Systems, Shanghai, China, 11–12 July 2011. FreeRTOS Project. Available online: http://www.freertos.org (accessed on 15 April 2017). Rabindra, P.K.; Kent, P. Rhealstone: A Real-Time Benchmarking Proposal. Available online: http:// collaboration.cmc.ec.gc.ca/science/rpn/biblio/ddj/Website/articles/DDJ/1989/8902/8902a/8902a.htm (accessed on 15 April 2017). ARM Generic Interrupt Controller Architecture Specification. Available online: http://www.cl.cam.ac.uk/ research/srg/han/ACSP35/zynq/arm_gic_architecture_specification.pdf (accessed on 15 April 2017). Simon, D. OS Porting & Analysis for Dual Core ARM Cortex-A9 Based Systems. Available online: http://www.imperas.com/documents/ARM_TechCon_2012_Imperas_Paper_OS_Porting_and_ Analysis_for_Cortex_A9MP_Based_Systems.pdf (accessed on 15 April 2017). Introduction to Basic RTOS Features Using SAM4L-EK FreeRTOS Port. Available online: http://ww1.microchip.com/downloads/en/DeviceDoc/Atmel-42247-Introduction-to-Basic%20RTOSFeatures-using-SAM4L-EK-FreeRTOS-Port_Training-Manual.pdf (accessed on 15 April 2017). Sustainability 2017, 9, 684 11. 12. 13. 14. 15. 16. 17. 18. 19. 14 of 14 Sung-Min, L.; Sang-Bum, S.; Bokdeuk, J.; Sangdok, M.; Brian Myungjune, J.; Jung-Hyun, Y.; Jae-Min, R.; Dong-Hyuk, L. Fine-grained I/O access control of the mobile devices based on the Xen architecture. In Proceedings of the 15th Annual International Conference on Mobile Computing and Networking, Beijing, China, 20–25 September 2009. Gernot, H.; Ben, L. The OKL4 Microvisor: Convergence Point of Microkernels and Hypervisors. Available online: https://ts.data61.csiro.au/publications/papers/Heiser_Leslie_10.slides.pdf (accessed on 15 April 2017). Gernot, H. Hypervisors for consumer electronics. In Proceedings of the 6th IEEE Conference on Consumer Communications and Networking Conference, Las Vegas, NV, USA, 11–13 January 2009. ARM Cortex-A15. Available online: https://en.wikipedia.org/wiki/ARM_Cortex-A15 (accessed on 15 April 2017). Francois, A.; Michel, G. A Practical Look at Micro-Kernels and Virtual Machine Monitors. In Proceedings of the 6th IEEE Conference on Consumer Communications and Networking Conference, Las Vegas, NV, USA, 10–13 January 2009. François, A.; Michel, G.; Gilles, M.; Gregory, M. Shared device driver model for virtualized mobile handsets. In Proceedings of the First Workshop on Virtualization in Mobile Computing (MobiVirt’08), Breckenridge, CO, USA, 17 June 2008. Keir, F.; Steven, H; Rolf, N.; Ian, P.; Andrew, W.; Mark, W. Safe Hardware Access with the xen Virtual Machine Monitor. Available online: http://www.cl.cam.ac.uk/research/srg/netos/papers/2004-oasis-ngio.pdf (accessed on 15 April 2017). Anderson, T.E.; Bershad, B.N.; Lazowska, E.D.; Levy, H.M. Scheduler Activations: Effective Kernel Support for the User-Level Management of Parallelism. Available online: http://flint.cs.yale.edu/cs422/doc/schedact.pdf (accessed on 15 April 2017). Video Clip for the Digital Timer Board in Experimental Results Section. Available online: https://youtu.be/ 4wMuyhlW97o (accessed on 15 April 2017). © 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
© Copyright 2026 Paperzz