We have upgraded the community system as part of the upgrade a password reset is required for all users before login in.

[Project] Docker Omega2 SDK for Cross Compilation + CMake support

  • @Jo-Kritzinger said in [Project] Docker Omega2 SDK for Cross Compilation:

    @José-Luis-Cánovas Yay! I got a hello world compiled, transferred and run on the omega. Thank you. seems there is light at the end of the tunnel.

    Then, lastly: I have both Omega2 and Omega2+, how can I set this up so I can develop for both of them? I noticed you mentioned editing some files but doe that mean I have to run a make overtime I change hardware platform?

    You shouldn't have to do much of anything different between the two when building ordinary userspace programs; only the bootloader and system builds really need to know about flash partitions, address width and memory size.

  • @Chris-Stratton Thank you Chris

  • @Jo-Kritzinger said in [Project] Docker Omega2 SDK for Cross Compilation:

    export PATH=$PATH:/lede/staging_dir/.... and so on
    and then
    export STAGING_DIR=/lede/staging_dir
    Is there a way of setting this without having to run the export commands so I don't have to do it every time?

    Add it to the /home/omega/.bashrc line, at the end, and those two export lines will execute everytime.

    I don't understand what you said about the -v option and not to do it. Docker is still a bit of a mystery to me. I use the /remote folder to copy the bin to so I can then connect via a terminal from the host machine to transfer it to the omega. I'ts a bit of a pain but I can live with it.

    That's actually a use for the -v option, to have a shared directory (/remote) between the container and your host machine. The same way you copy your bin to /remote and then you can access it from your mac, use the other way around to copy your source code to the machine. That way you can edit your code with Atom/Sublime text/XCode/etc. from your mac, and access them from the container in /remote.

    Also, lets say you want to use the i2c library. So, you specify the #include <onion-i2c.h> in the source, but how do you compile/link and generate the binary file? Or, how would you get to the gpio's? Would you use a pointer to the addresses in memory directly? A long * ?

    I haven't used the I2C library, but here is the official documentation. In it you can read:

    Library for Linker
    In your project’s makefile, you will need to add the following dynamic libraries to the linker command:
    -loniondebug -lonioni2c
    The dynamic libraries are stored in /usr/lib on the Omega.

    The two -l options you add to the Makefile are the same you add to your gcc command. It tells to find dynamic libraries named oniondebug and onioni2c, but doesn't need them now (dynamic). The library files are already inside your omega2, so you don't need to copy any files.

    What are the most common options you specify on the mipsel-openwrt-linux-gcc line?

    The same I would use with common gcc. Like before, if I need a dynamic library, use the -l option, etc. Depends on your project. You can search for an introduction to Makefile for "good practices", but it can became something too big for what you want, so stay in the basis of make.

    Then, lastly: I have both Omega2 and Omega2+, how can I set this up so I can develop for both of them? I noticed you mentioned editing some files but does that mean I have to run a make eveyrtime I change hardware platform?

    I don't know exactly what processor has the Omega2+, but I don't think it's different from the MIPSel of the Omega2. To build executable binaries the same compiler should do fine, try to run the same binary hello_world in both and check for yourself. To build your own firmware, then you must proceed with caution not to break your omega.

    Thank you José. I really appreciate your help.

    No worries. This messages can help other people too, and we all started from the bottom.

  • Thank you José.

  • @Jo-Kritzinger
    if you are still interested, I added (in the OP edits) the image with all the make commands to build the SDK already executed, and also added CMake support, very easy to use and works like a charm for me.

    If you use CLion projects for example, you only have to copy your source code to the shared directory (between your mac and docker), copy the Toolchain-omega2-mipsel.cmake file I posted in the Edit 3 and run cmake from docker as said in the last lines.
    If any errors happen, delete the CMake cache files and try again.

  • @José-Luis-Cánovas thank you José

  • @José-Luis-Cánovas José, me again. I used the -lonioni2c etc. but it says it can not find the libraries.

    /lede/staging_dir/toolchain-mipsel_24kc_gcc-5.4.0_musl-1.1.16/lib/gcc/mipsel-openwrt-linux-musl/5.4.0/../../../../mipsel-openwrt-linux-musl/bin/ld: cannot find -loniondebug
    /lede/staging_dir/toolchain-mipsel_24kc_gcc-5.4.0_musl-1.1.16/lib/gcc/mipsel-openwrt-linux-musl/5.4.0/../../../../mipsel-openwrt-linux-musl/bin/ld: cannot find -lonionspi
    collect2: error: ld returned 1 exit status

    Then I added in the -L/usr/lib. Same deal. I added a -v on the line and, somewhere in the heap of messages, I see "Ignoring nonexistent directory "/lede/staging_dir/usr/include". So, from my limited experience, it seems that the linker needs

    • to see the actual libraries at compile/link time
    • is looking at a directory in docker, i.e. /lede/staging_dir

    Is this because of using Docker and does it mean I will have to download the actual source files and compile those, ignoring the actual library?

    By the by, there is something odd in this whole thing as the h files are missing so I was forced to download those as well. Logically (again in my limited experience in the Linux world) I would expect the see the h files in the same (or similar) directory as the actual library .so file? e.g. /usr/include or something? I mean, the pre-processor and compiler must have access to them right? Well, really, they (h and so files) should be part of Docker?

    I feel I am missing something here.

    Probably that I do not understand the whole Docker setup properly as Docker is (obviously) not aware of the actual hardware/firmware/software that is on the Omega meaning referencing any directories are not going to work as expected because the compilation/linking is not taking place on the target system?

    Does using docker mean that using shared object libraries are off the cards?

    In the interest of completeness, this is the last gcc instruction I used:
    mipsel-openwrt-linux-gcc -L/usr/lib -Wl,-rpath=/usr/lib -Wall testSpi.c -loniondebug -lonionspi -o testspi.out

  • @Jo-Kritzinger
    Think of Docker as a virtual machine (doesn't work like that, but for the sake of simplicity), in particular, an Ubuntu 14.04 with some files added (/lede/*).

    Anything you run on the Docker container, doesn't affect your host machine, only your shared directories, and those are for copying code.

    Now that we think of the container as a machine on its own, let's see the SDK.
    Ubuntu comes with gcc that compiles C code to its own architecture, and it can run the compiled binaries with its shared libraries, e.g. in /usr/lib. All that is fine, if you compile for the PC.
    Omega2 can't compile anything because the compiler is too big for it to run on the Omega2. Therefore, we need a cross compiler, including the gcc binary and cross compiled libraries aiming for the same MIPSel arquitecture as the Omega2 has.
    The gcc binary and cross compiled libraries are not in your Ubuntu 14.04 /usr/lib directory, because Ubuntu doesn't need any of them. They are inside the /lede/staging_dir/toolchain-mipsel_24kc_gcc-5.4.0_musl/ (the compiler) and /lede/staging_dir/target-mipsel_24kc_musl/ (the include and library files) directories. There is where mipsel-openwrt-linux-gcc must look for the headers and libraries (see below).

    Your errors come from the linker not finding suitable files for your specified libraries, because the directory is wrong.

    One bug I find is that your toolchain dir has a -1.1.16 in the end. It is now gone in the new builds of the docker image.

    Another thing you can do is pasting these lines in your ~/.bashrcfile, or exec a docker pull from the last image built, where the env vars are already set:

    export PATH "$PATH:/lede/staging_dir/toolchain-mipsel_24kc_gcc-5.4.0_musl/bin"
    export STAGING_DIR "/lede/staging_dir"
    export CFLAGS=-I/lede/staging_dir/target-mipsel_24kc_musl/usr/include
    export LDFLAGS=-L/lede/staging_dir/target-mipsel_24kc_musl/usr/lib

    Notice that PATH no longer has -1.1.16, and the include and libraries directories are set for the compiler.

    Finally, once you compile successfully and copy it to the Omega2, the binary that uses shared libraries will find the .so objects in the Omega2 too, but the lede SDK already have them all in your host machine.

  • @José-Luis-Cánovas thank you José, I will download the new image and try again. Will it install overvthe old image or must I use docker to delete the old one?

  • @José-Luis-Cánovas Still having issues. I pulled the jlcs/omega2-docker-built and ran with
    run -it --name omega2-sdk-app-f -v /<inserted my host dir here>:/remote jlcs/omega2-sdk bash.
    (NOTE the --name omega2-sdk-app-f. Without it I got a error saying it exists already and I was not to keen to delete because I'm not familiar with the system and was worried I'd delete something important so I just gave it a new name)

    This gave me a omega@a1e6ec1ab9d3: prompt.

    Then I tried a compile and it failed. So, I investigated a bit further:

    printenv results in the following:
    LS_COLORS(====Edit all the colour info out=====)
    no_proxy=*.local, 169.254/16
    LESSOPEN=| /usr/bin/lesspipe %s
    LESSCLOSE=/usr/bin/lesspipe %s %s

    Then I did this to see if I could spot something:
    result ==> /lede/staging_dir/target-mipsel_24kc_musl

    result ==> bin include lib stamp
    *NOTE, There is no /usr directory

    cd include/
    result ==>

    cd ..
    cd lib/
    ls -a
    Result ==> . ..

    Here is the compile instruction:
    /lede/staging_dir/target-mipsel_24kc_musl/lib$ mipsel-openwrt-linux-gcc -Wall /remote/testSpi.c -lonionspi -o testSpi.o
    /lede/staging_dir/toolchain-mipsel_24kc_gcc-5.4.0_musl/lib/gcc/mipsel-openwrt-linux-musl/5.4.0/../../../../mipsel-openwrt-linux-musl/bin/ld: cannot find -lonionspi
    collect2: error: ld returned 1 exit status

    So I ran the compile with the -v again and the same line came up again in the whole list:
    ignoring nonexistent directory "/lede/staging_dir/usr/include"

    Lastly I did:
    find /lede -name *.h | grep libonion


    What am I doing wrong?

  • @Jo-Kritzinger The image will update, but you will have to run a new container from it with docker run ....

  • @Jo-Kritzinger said in [Project] Docker Omega2 SDK for Cross Compilation + CMake support:

    I got a error saying it exists already and I was not to keen to delete because I'm not familiar with the system and was worried I'd delete something important so I just

    It is the old container. In Docker we have images and containers, images are static, and containers dynamic, what you run. With your command you are telling docker to create a new container with a fixed name; because it already exists (the container you have been working with) it shows an error.

    I pulled the jlcs/omega2-docker-built and ran with
    run -it --name omega2-sdk-app-f -v /<inserted my host dir here>:/remote jlcs/omega2-sdk bash.

    Here is the error. The penultimate argument in your command should be jlcs/omega2-docker-built, the name of the new image, but you used jlcs/omega2-sdk, the one that didn't run the make commands.

  • @José-Luis-Cánovas Thanks José, I will give it a go tomorrow. I shouls have spotted that. (Embarrased grin)

  • @José-Luis-Cánovas
    thank you for your work, the container runs perfect and you saved me a lot of time!

  • @Philipp-Zeitschel you are welcome! ❤

  • This post is deleted!

  • @José-Luis-Cánovas Hi José. I have been doing other stuff and am just getting back into this.
    I'm still missing something. I think I have the right one running, checked by doing a docker ps that resulted in

    628a8ef6b111 jlcs/omega2-docker-built "bash" 27 minutes ago Up 3 minutes omega2-sdk-app-built

    Also /lede/staging_dir/target-mipsel_24kc_musl/include and /lib are both empty.

    This is the result from the compiler:

    mipsel-openwrt-linux-gcc   -Wall testSpi.c -loniondebug -lonionspi -o testspi.out
    /lede/staging_dir/toolchain-mipsel_24kc_gcc-5.4.0_musl/lib/gcc/mipsel-openwrt-linux-musl/5.4.0/../../../../mipsel-openwrt-linux-musl/bin/ld: cannot find -loniondebug
    /lede/staging_dir/toolchain-mipsel_24kc_gcc-5.4.0_musl/lib/gcc/mipsel-openwrt-linux-musl/5.4.0/../../../../mipsel-openwrt-linux-musl/bin/ld: cannot find -lonionspi
    collect2: error: ld returned 1 exit status

    Any advise?

    Thank you.

  • @Jo-Kritzinger yes, silly me, I saw the directories errors, but not that you wanted onion libraries.

    [Run inside the docker container]
    You must add the onion feeds (work in progress for my docker image, in the meantime do it manually) to the file feeds.conf.default (check WereCatf's first_time_setup.sh), update the feed and add the onion packages:
    scripts/feeds update -a
    scripts/feeds install -a
    Then, from make menuconfig select the onion libraries (Onion -> Libraries/Utilities -> select as [M]odule).
    Don't worry about a worning with recursive dependencies.
    Run this chunk of code to avoid adding an SSH certificate to github only to clone some Onion packages:

    (Read Edit 1)

    # From https://github.com/WereCatf/source/blob/image/first_time_setup.sh
    #Ugly fix for several packages
    #Onion-devs don't give a fuck
    find feeds/onion -iname "Makefile" | while read filename
    githubUrl=$(grep "git@github.com" "$SCRIPTDIR/$filename")
    if [ $? -eq 0 ]
        githubUrl=$(echo "$githubUrl"|sed 's/git@github.com:/https:\/\/github.com\//'|sed 's/\.git//'|cut -d '=' -f 2-)
        curl --output /dev/null --silent --head --fail "$githubUrl"
        if [ $? -eq 0 ]
            echo "Patching $filename..."
            sed -i 's/git@github.com:/https:\/\/github.com\//' "$SCRIPTDIR/$filename"

    Then run make -j#, where # is your number of cores +1, to speed things up (check again the end of first_time_setup.sh).

    Whenmake finishes, an ls staging_dir/target-mipsel_24kc_musl/usr/lib/ (note the penultimate /usr/ directory) shows:

    omega@dee42fad6a75:/lede$ ls staging_dir/target-mipsel_24kc_musl/usr/lib/
    [...]    liboniondebug.so   [...]    libonioni2c.so  [...]    libonionspi.so  [...]

    and many more onion and non-onion libraries.

    And inside staging_dir/target-mipsel_24kc_musl/usr/include/:

    omega@dee42fad6a75:/lede$ ls staging_dir/target-mipsel_24kc_musl/usr/include/                                         
    [...] onion-spi.h   [...]    onion-debug.h    [...]      onion-i2c.h      [...]
    Edit 1: Install curl before running the script:

    First, we need to login as root in the container to gain privileges to install new packages.
    With the docker container stopped, run:
    [Run in your host's terminal (mac, windows, linux)]

    $   docker start omega2-sdk-app-built
    $   docker exec -ti --user root omega2-sdk-app-built bash

    Now you are root user inside the container. Install the curl package.
    [Run inside the docker container]

    #   apt update && apt install -y curl
    #   exit

    Then, go back to the container as omega user with the usual:
    [Run in your host's terminal (mac, windows, linux)]

    $   docker start omega2-sdk-app-built
    $   docker attach omega2-sdk-app-built

    Run the script above.

  • @José-Luis-Cánovas Haha, no worries. When I have a block of time I'll sit down and give this a go. I'm assuming this is all run (esp the first chunk of code) from within docker, i.e. at the omega@xxxxxx prompt?

    By the way, I really appreciate your help.

  • @Jo-Kritzinger yes, everything inside the docker container except for the docker commands in Edit 1.
    The omega2 will have the headers and libraries already inside, and if not, use opkg update and opkg install <package> where package is your desired onion library. In the SDK, selecting with make menuconfig and building with make compiles the source code of these libraries, generating the necessary files for the linker (ld).

Looks like your connection to Community was lost, please wait while we try to reconnect.