分享到plurk 分享到twitter 分享到facebook

版本 8bbe5ff4e7cf37b27b675cdcda07a85ab3999636

Xenomai

建立環境

  • Install Cross complier

.. code-block:: c

cd <working dir>
wget https://github.com/raspberrypi/tools/archive/master.tar.gz
tar xzf master.tar.gz
  • Download source files and patches

    • Download kernel

    .. code-block:: c

    git clone -b rpi-3.8.y --depth 1 git://github.com/raspberrypi/linux.git linux-rpi-3.8.y
    • Download Xenomai

    .. code-block:: c

    git clone git://git.xenomai.org/xenomai-head.git xenomai-head
    • Download minimal config

    .. code-block:: c

    wget https://www.dropbox.com/s/dcju74md5sz45at/rpi_xenomai_config
  • Apply patches

    • Apply ipipe core pre-patch

    .. code-block:: c

    cd linux-rpi-3.8.y patch -Np1 < ../xenomai-head/ksrc/arch/arm/patches/raspberry/ipipe-core-3.8.13-raspberry-pre-2.patch

    • Apply Xenomai ipipe core patch

    .. code-block:: c

    xenomai-head/scripts/prepare-kernel.sh –arch=arm –linux=linux-rpi-3.8.y –adeos=xenomai-head/ksrc/arch/arm/patches/ipipe-core-3.8.13-arm-3.patch

    • Apply ipipe core post-patch

    .. code-block:: c

    cd linux-rpi-3.8.y patch -Np1 < ../xenomai-head/ksrc/arch/arm/patches/raspberry/ipipe-core-3.8.13-raspberry-post-2.patch

  • Compile kernel

    • Create build directory

    .. code-block:: c

    mkdir linux-rpi-3.8.y/build

    • Configure kernel

    .. code-block:: c

    cp rpi_xenomai_config linux-rpi-3.8.y/build/.config cd linux-rpi-3.8.y make mrproper make ARCH=arm O=build oldconfig

    • Compile

    .. code-block:: c

    make ARCH=arm O=build CROSS_COMPILE=../../tools-master/arm-bcm2708/arm-bcm2708hardfp-linux-gnueabi/bin/arm-bcm2708hardfp-linux-gnueabi-

    • Install modules

    .. code-block:: c

    make ARCH=arm O=build INSTALL_MOD_PATH=dist modules_install

    • Install headers

    .. code-block:: c

    make ARCH=arm O=build INSTALL_HDR_PATH=dist headers_install find build/dist/include ( -name .install -o -name ..install.cmd ) -delete

  • 編譯好的kernelImage,移到SD卡的 /boot/ 路徑下並更改名稱為kernel.img

  • linux-rpi-3.8.y/build/dist中的Module,移到SD卡中的/lib/modules

  • Cyclictest

    • Linux

    .. code-block:: c

    • Xenomai

    .. code-block:: c

    cd xenomai-head export PATH=../tools-master/arm-bcm2708/arm-bcm2708hardfp-linux-gnueabi/bin/:$PATH ./configure –host=arm-bcm2708hardfp-linux-gnueabi cd src mkdir dist make install DIST_DIR=dist

    dist中會出現usr/xenomai 將這個資料夾移到sd卡中 /usr/

    在raspberry pi中

    .. code-block:: c

    export PATH=/usr/xenomai/bin:$PATH
    export LD_LIBRARY_PATH=/usr/xenomai/lib sudo modprobe xeno_posix

    接著就能跑使用xenomai機制的cyclictest

    • RT_preempt

Real Time 的定義

  • Hard Real Time

系統一定可以在 Response Time 內完成指定的task

  • Soft Real Time

在特定的機率下,系統可以在 Response Time 內完成指定的task

作業系統架構

.. image:: /xenoarch2.jpg

Xenomai是一個linux kernel的patch 藉由在底層增加一個架構 負責硬體與接收interrupt 並將interrupt 傳給上層的OS(這邊稱為domain)

這個底層的架構是Adeos 是另一個open source的project

基本架構示意圖 http://dchabal.developpez.com/tutoriels/linux/xenomai/images/image04.png

在api呼叫上可以看到不同層級的抽象化 ipipe_XXX -> rthal_XXX -> xnXXX 上一層對下一層的操作方式基本上會固定 但下一層的實作一硬體或config有所差異

負責傳送interrupt的程式稱為ipipe 示意圖 http://www.xenomai.org/documentation/xenomai-2.6/html/pictures/life-with-adeos-img4.jpg

可以找到ipipe_raise_irq()將interrupt推到pipeline

在ipipe上每個domain都有自己的優先度 高優先度的domain會先接收到interrupt 高優先度的domain的thread 可以preempt 低優先度domain的thread

iPipe ++++++++++++++

主要負責處理irq 與 timer(HRT), ipipe的工作很簡單 就是設定timer並將interrupt往上丟

  • 相關檔案︰

    • gic.c :

      Generic Interrupt Controller, Interrupt prioritization and distribution to each CPU interface. This is known as the Distributor. Priority masking and preemption handling for each CPU. This is known as the CPU Interface.

    • it8152.c:IRQ相關

    • timer-sp.c:dual timer module(sp804)

    • vic.c:

      The VIC provides a software interface to the interrupt system. In a system with an interrupt controller, software must determine the source that is requesting service and where its service routine is loaded. A VIC does both of these in hardware.

    • ipipe-tsc.c:設定精準度(刻度)

    • ipipe/compat.c:interrupt

    • sched/clock.c:取得cpu_clock 解析度為nanasecond,開機後從0開始上數

HAL ++++++++++++

Hardware Abstract Layer:prcess 透過HAL呼叫ipipe的服務。這一層主要是包裝ipipe 與底層資訊 讓nucleus可以不用看到硬體資訊。

Nucleus ++++++++++++

Xenomai的kernel, 包含一個scheduler,優先執行real-time tasks.

Scheduler ++++++++++++

優先處理realtime task ,linux也被視為其中一個thread,本身也有scheduler,但須等到沒有real-time task時(idle state),才會執行linux thread

Skins ++++++++++++

呼叫xenomai的界面, 有native rtdm posix等。

問題 ++++++++++++

與 RT-PREEMPT 途徑的差異?

實作

Context switch ++++++++++++++

.. code-block:: prettyprint

include/arch/arm-asm/bits/pod.h
static inline void xnarch_switch_to(xnarchtcb_t *out_tcb, xnarchtcb_t *in_tcb)

{

        struct task_struct *prev = out_tcb->active_task;

        struct mm_struct *prev_mm = out_tcb->active_mm;

        struct task_struct *next = in_tcb->user_task;

        if (likely(next != NULL)) {

                in_tcb->active_task = next;

                in_tcb->active_mm = in_tcb->mm;

                rthal_clear_foreign_stack(&rthal_domain);

        } else {

                in_tcb->active_task = prev;

                in_tcb->active_mm = prev_mm;

                rthal_set_foreign_stack(&rthal_domain);

        }

        if (prev_mm != in_tcb->active_mm) {

                /* Switch to new user-space thread? */

                if (in_tcb->active_mm)

                        wrap_switch_mm(prev_mm, in_tcb->active_mm, next);

                if (!next->mm)

                        enter_lazy_tlb(prev_mm, next);

        }

        /* Kernel-to-kernel context switch. */

        rthal_thread_switch(prev, out_tcb->tip, in_tcb->tip);

}

.. code-block:: prettyprint

ksrc/arch/arm/switch.S
/*
/*
 * Switch context routine.
 *
 * Registers according to the ARM procedure call standard:
 *   Reg    Description
 *   r0-r3  argument/scratch registers
 *   r4-r9  variable register
 *   r10=sl stack limit/variable register
 *   r11=fp frame pointer/variable register
 *   r12=ip intra-procedure-call scratch register
 *   r13=sp stack pointer (auto preserved)
 *   r14=lr link register
 *   r15=pc program counter (auto preserved)
 *
 * Copied from __switch_to, arch/arm/kernel/entry-armv.S.
 * Right now it is identical, but who knows what the
 * future reserves us...
 *
 * XXX: All the following config options are NOT tested:
 *      CONFIG_IWMMXT
 *
 *  Calling args:
 * r0 = previous task_struct, r1 = previous thread_info, r2 = next thread_info
 */
ENTRY(rthal_thread_switch)
        add     ip, r1, #TI_CPU_SAVE
 ARM(        stmia        ip!, {r4 - sl, fp, sp, lr} )        @ Store most regs on stack
 THUMB(        stmia        ip!, {r4 - sl, fp}           )        @ Store most regs on stack
 THUMB(        str        sp, [ip], #4                   )
 THUMB(        str        lr, [ip], #4                   )
        load_tls r2, r4, r5
#ifdef USE_DOMAINS
    ldr     r6, [r2, #TI_CPU_DOMAIN]
#endif
        clear_exclusive_monitor
        switch_tls r1, r4, r5, r3, r7
#ifdef USE_DOMAINS
        mcr     p15, 0, r6, c3, c0, 0           @ Set domain register
#endif
        fpu_switch r4
 ARM(        add        r4, r2, #TI_CPU_SAVE           )
 ARM(        ldmia        r4, {r4 - sl, fp, sp, pc}  )        @ Load all regs saved previously
 THUMB(        add        ip, r2, #TI_CPU_SAVE           )
 THUMB(        ldmia        ip!, {r4 - sl, fp}           )        @ Load all regs saved previously
 THUMB(        ldr        sp, [ip], #4                   )
 THUMB(        ldr        pc, [ip]                   )
ENDPROC(rthal_thread_switch)

Timer +++++++++++

Scheduler +++++++++++ 依據預設 Xenomai 組態,nucleus 程式碼的兩個檔案未被編譯進去: - sched-tp.c - sched-sporadic.c

xnpod +++++++++++

.. code-block:: prettyprint

struct xnpod {

xnflags_t status;	/*!< Status bitmask. */

xnsched_t sched[XNARCH_NR_CPUS];	/*!< Per-cpu scheduler slots. */

xnqueue_t threadq;	/*!< All existing threads. */
#ifdef CONFIG_XENO_OPT_VFILE
struct xnvfile_rev_tag threadlist_tag;
#endif
xnqueue_t tstartq,	/*!< Thread start hook queue. */
tswitchq,		/*!< Thread switch hook queue. */
tdeleteq;		/*!< Thread delete hook queue. */

atomic_counter_t timerlck;	/*!< Timer lock depth.  */

xntimer_t tslicer;	/*!< Time-slicing timer for aperiodic mode  */
int tsliced;		/*!< Number of threads using the slicer */

int refcnt;		/*!< Reference count.  */

#ifdef __XENO_SIM__
void (*schedhook) (xnthread_t *thread, xnflags_t mask);	/*!< Internal scheduling hook. */
#endif	/* __XENO_SIM__ */
};

Cyclictest +++++++++++

.. code-block:: prettyprint

實驗數據

.. code-block:: prettyprint

 pi@raspberrypi:~$ cat /proc/xenomai/stat
 CPU    PID        MSW                  CSW           PF         STAT            %CPU    NAME
 0      0            0                  206           0          00500080        100.0   ROOT
 0      0            0                  2688553       0          00000000        0.0     IRQ3: [timer]

.. code-block:: prettyprint

 pi@raspberrypi:~$ sudo /usr/xenomai/bin/cyclictest >/dev/null 2>/dev/null &
 [1] 2253

.. code-block:: prettyprint

 pi@raspberrypi:~$ ps aux | grep -i "cy"
 root      2253  0.5  0.3   4580   1464  ?        S    03:34   0:00   sudo /usr/xenomai/bin/cyclictest
 root      2254  2.7  0.4   2340   2132  ?        SLl  03:34   0:00   /usr/xenomai/bin/cyclictest
 pi        2259  0.0  0.1   3540   820   ttyAMA0  S+   03:34   0:00   grep --color=auto -i cy

.. code-block:: prettyprint

 pi@raspberrypi:~$ cat /proc/xenomai/stat
 CPU    PID        MSW                CSW              PF        STAT        %CPU    NAME
 0      0          0                  255              0         00500080    100.0   ROOT
 0      2254       1                  1                0         00b00380    0.0     cyclictest
 0      2256       2                  48               0         00300184    0.0     cyclictest
 0      0          0                  2913946          0         00000000    0.0     IRQ3: [timer]

.. code-block:: prettyprint

 pi@raspberrypi:~$ watch -n 1 cat /proc/xenomai/stat
 Every 1.0s: cat /proc/xenomai/stat                      Wed Jan  8 03:38:43 2014

 CPU    PID        MSW                CSW           PF        STAT         %CPU     NAME
 0      0          0                  442           0         00500080     99.9     ROOT
 0      2254       1                  1             0         00b00380     0.0      cyclictest
 0      2256       2                  235           0         00300184     0.0      cyclictest
 0      0          0                  2953543       0         00000000     0.1      IRQ3: [timer]

效能表現

  • Cyclictest<https://rt.wiki.kernel.org/index.php/Cyclictest>_
  • Test case: POSIX interval timer, Interval 500 micro seconds,. 100000 loops, 100% load.
    • Commandline: cyclictest -t1 -p 80 -i 500 -l 100000
  • 使用 PREEMPT LINUX

.. code-block:: prettyprint

root@raspberrypi:/home/pi# sudo ./cyclictest -t1 -p 80 -i 500 -l 100000
# /dev/cpu_dma_latency set to 0us
policy: fifo: loadavg: 0.00 0.01 0.05 1/61 2064          
T: 0 ( 2063) P:80 I:500 C: 100000 Min:     27 Act:   49 Avg:   42 Max:    1060
  • 使用 RT-PREEMPT

.. code-block:: prettyprint

Linux raspberrypi 3.6.11+ #474 PREEMPT Thu Jun 13 17:14:42 BST 2013 armv6l GNU/Linux
Min:     22 Act:   31 Avg:   32 Max:     169
  • 使用 Xenomai

.. code-block:: prettyprint

Linux raspberrypi 3.8.13-core+ #1 Thu Feb 27 03:02:16 CST 2014 armv6l GNU/Linux
Min:      1 Act:    5 Avg:    6 Max:      41

.. code-block:: prettyprint

root@raspberrypi:/home/pi# /usr/xenomai/bin/cyclictest -t1 -p 80 -i 500 -l 10000 
0.08 0.06 0.05 1/61 2060          
T: 0 ( 2060) P:80 I:     500 C:  100000 Min:      -4 Act:      -2 Avg:       0 Max:      30
  • cyclictest 做法

概念: 取得現在時間接著讓process睡一個間隔 等process醒來後再取一次時間 比對兩次取得的時間差與設定的間隔差距

pseudocode:

.. code-block:: prettyprint

clock_gettime((&now))
next = now + par->interval
while (!shutdown) {
    clock_nanosleep((&next))
    clock_gettime((&now))
    diff = calcdiff(now, next)
    # update stat-> min, max, total latency, cycles
    # update the histogram data
    next += interval
}

造成這時間差的可能原因:

timer精準度

interrupt latency

interrupt handler duration

scheduler latency

scheduler duration

context switch

討論用

https://embedded2014.hackpad.com/Xenomai-raspberry-note-XwJtuQn9nkD

組員

  • 向澐
  • 林家宏
  • 呂科進
  • 趙愷文
  • 阮志偉
  • 陳建霖

參考資料

  • https://code.google.com/p/picnc/wiki/RPiXenomaiKernel
  • https://code.google.com/p/picnc/wiki/CreateRaspbianLinuxCNC
  • http://www.camelsoftware.com/firetail/blog/raspberry-pi/real-time-operating-systems/
  • Quadruped Linux robot feels its way over obstacles<http://linuxgizmos.com/hyq-quadruped-robot-runs-real-time-linux/>_
  • Choosing between Xenomai and Linux for real-time applications<https://www.osadl.org/fileadmin/dam/rtlws/12/Brown.pdf>_
  • Real Time Systems<http://www.slideshare.net/anil_pugalia/real-time-systems>_
  • http://www.cs.ru.nl/lab/xenomai/exercises/