Cross compile Mono from source for the Omega2 and Omega2+

  • Background info

    In this thread I will explain how to compile the latest Mono from source for the Onion Omega2 and Omega2+, including some things to customize Mono for your needs.
    Mono compilation consists of two things

    • Native code (C Code). The main program, virtual execution system, garbage collector etc.
    • Managed code (C# Code). The Mono Library.

    The first thing you will need to compile is the main program written in C. After you have done this, you can compile the Library (C#). Under normal circumstances you can do both in one make command. That only works when you are compiling on the device itself. In this tutorial I will use a cross toolchain to compile the main program for the Omega2. Mono does not support compiling managed code when cross compiling. For that you will need the Linux host on which you are cross compiling or a different Linux host to compile the managed code. After compilation is done you can just copy the C# library from the Linux host.

    Used Software

    I used the following Linux distro for cross compiling

    • Linux host: Ubuntu Server 16.04 LTS

    I used the following cross toolchain for cross compiling

    Step 1. Compile Mono on the Linux host

    I used the following working directory

    • /home/project/

    The first step is to compile Mono on the Linux host. Go to the working directory

    cd /home/project/

    Download and install all the necessary packages

    sudo apt-get install subversion g++ zlib1g-dev build-essential git python libncurses5-dev gawk gettext unzip file libssl-dev wget libtool-bin autoconf cmake

    Clone the mono source

    git clone –recursive

    Change to mono directory

    cd mono

    Configure and make mono

    ./ --prefix=/ && make

    Create directory for exporting mono installation

    mkdir ../BuildHost

    Make install Mono

    make install DESTDIR=/home/project/BuildHost

    Now you have a full working Mono installation for your Linux host. We only need the library /home/project/BuildHost/lib/mono

    Clean up source directory

    make clean

    Step 2. Cross Compile mono for Omega2

    Then we will build the main program for the Omega2.
    Create the cross toolchain environment. For that you will need the cross toolchain binaries from the LEDE SDK directory. In my case /home/project/lede-sdk-17.01.0-ramips-mt7688_gcc-5.4.0_musl-1.1.16.Linux-x86_64/staging_dir/toolchain-mipsel_24kc_gcc-5.4.0_musl-1.1.16/bin/

    export CROSS=/home/project/lede-sdk-17.01.0-ramips-mt7688_gcc-5.4.0_musl-1.1.16.Linux-x86_64/staging_dir/toolchain-mipsel_24kc_gcc-5.4.0_musl-1.1.16/bin/mipsel-openwrt-linux
    export CC=$CROSS-gcc
    export CXX=$CROSS-g++
    export CPP=$CROSS-cpp
    export AR=$CROSS-ar
    export AS=$CROSS-as

    Ok so the cross toolchain environment is set up. At this point you will need to tweak the configure script so that it forces to use __thread as Thread Local Storage for Mono. The configure script automatically selects pthread instead of __thread. The reason for doing this is because the standard garbage collector for mono, sgen does not support pthread. Normally you can select boehm as the alternative garbage collector, but since boehm doesn’t support the musl library used for the Omega2 there is no other choice then to tweak the configure script. Also using the option --with-tls=__thread with the configure script doesn’t work. So tweaking it is. Fortunately __thread works just fine on the Omega2.

    Tweak the configure script

    sed -i -- 's/with_tls=pthread/with_tls=__thread/g' configure 

    What I could not figure out was this error when I was cross compiling Mono

    In file included from mono-context.c:10:0:
    mono-context.c: In function 'mono_sigctx_to_monoctx':
    ../../mono/utils/mono-sigcontext.h:480:73: error: request for member 'fp_r' in something not a structure or union
     # define UCONTEXT_FPREGS(ctx) (((ucontext_t *)(ctx))->uc_mcontext.fpregs.fp_r.fp_dregs)
    mono-context.c:448:24: note: in expansion of macro 'UCONTEXT_FPREGS'
       mctx->sc_fpregs[i] = UCONTEXT_FPREGS (sigctx) [i];
    mono-context.c: In function 'mono_monoctx_to_sigctx':
    ../../mono/utils/mono-sigcontext.h:480:73: error: request for member 'fp_r' in something not a structure or union
     # define UCONTEXT_FPREGS(ctx) (((ucontext_t *)(ctx))->uc_mcontext.fpregs.fp_r.fp_dregs)
    mono-context.c:460:3: note: in expansion of macro 'UCONTEXT_FPREGS'
       UCONTEXT_FPREGS (sigctx) [i] = mctx->sc_fpregs[i];

    If you have suggestions on how to fix this, comment it please. I have been working on this error for quite some time. It has to do with the musl library used with the Mips architecture. It compiles normally when glibc library is used.

    For now just remove the lines which gives the make error.

    sed -i '/# define UCONTEXT_FPREGS(ctx)/c\' mono/utils/mono-sigcontext.h
    sed -i '/UCONTEXT_FPREGS (sigctx)/c\' mono/utils/mono-context.c

    Then you can configure Mono. Because boehm doesn’t work we can disable it. At this point you can select which features you want to use with Mono. I highly recommend to create a small footprint since the Omega2 does not have much space. You can of course expand the root partition with a USB flash drive or a MicroSD card and use a completer version of Mono. In this tutorial I will explain how to build a small footprint

    Configure mono with small footprint

    ./ --prefix=/ --enable-minimal=aot,profiler,debug,logging --host=mipsel-openwrt-linux --enable-small-config=yes --with-shared_mono=no --with-static_mono=yes --disable-mcs-build --without-mcs-docs --with-moonlight=no --disable-boehm

    Then make mono with optimization for size

    make CFLAGS=-Os

    Create directory for exporting Omega2 Mono installation

    mkdir ../BuildOmega

    Export Omega2 Mono installation

    make install DESTDIR=/home/project/BuildOmega

    Change to BuildOmega directory

    cd ../BuildOmega

    Remove include and share. We don’t need those

    rm -r include share

    Change to bin directory

    cd bin

    Remove all binaries except mono and mono-sgen

    mv mono mono-sgen ../
    rm *
    mv ../mono ../mono-sgen .

    Then strip the mono binary

    $CROSS-strip mono-sgen

    Change to lib directory

    cd ../lib

    Strip some libraries

    $CROSS-strip lib*

    Remove unnecessary files and directories

    rm *.la *.a
    rm -r pkgconfig mono

    Now we can copy the .NET library to the Mono Omega build directory
    If size doesn’t matter (chuckle), just merge the whole directory /home/project/BuildHost/lib/mono with /home/project/BuildOmega/lib/mono so that it has the complete mono library.
    Or just choose which .NET features you want to use and copy those to /home/project/BuildOmega/lib/mono

    I will explain how to create a very small library, almost equal to Debian’s mono-runtime

    cd /home/project/BuildHost/lib/mono
    mv 4.5 gac ../
    rm -r *
    mv ../4.5 ../gac .
    cd 4.5
    mv cert-sync.exe gacutil.exe I18N.dll I18N.West.dll Mono.Security.dll mscorlib.dll System.Configuration.dll System.dll System.Security.dll System.Xml.dll ../
    rm -r *
    mv ../cert-sync.exe ../gacutil.exe ../I18N.dll ../I18N.West.dll ../Mono.Security.dll ../mscorlib.dll ../System.Configuration.dll ../System.dll ../System.Security.dll ../System.Xml.dll .
    cd ../gac
    mv I18N I18N.West Mono.Security System System.Configuration System.Security System.Xml ../
    rm -r *
    mv ../I18N ../I18N.West ../Mono.Security ../System ../System.Configuration ../System.Security ../System.Xml .
    rm I18N/*/*.pdb I18N.West/*/*.pdb Mono.Security/*/*.pdb System/*/*.pdb System.Configuration/*/*.pdb System.Security/*/*.pdb System.Xml/*/*.pdb

    Then copy the library to the BuildOmega directory

    cp -R /home/project/BuildHost/lib/mono /home/project/BuildOmega/lib/mono

    You can then copy the contents of /home/project/BuildOmega to the root of your Omega2(+).

    This is how you compile Mono for the Omega2. If there are any questions, please ask them in the comments.

  • Thanks for sharing this info, very valuable!

    Is there a chance to share also some more info regarding your C# projects that you run with this mono (.NET version, environment, compiling, library compatibility issues, etc)?
    Would love to hear more on C# on Omega Onion!

    I wonder is it achievable to build a .NET app using Visual Studio (.NET Core?) that could handle HTTP requests (Kestrel/WebListener or even HttpServer for non-core), manage Omega GPIO and I2C bus as well as managing other native OpenWRT processes outside the mono (e.g. madplay).