[Solved, more or less] OpenWRT C++ CrossCompile uClibc setup

  • Hiho,
    I want to compile a "helloWorld"-package with the openwrt standart setup found on: https://github.com/OnionIoT/OpenWRT-Packages/wiki/Setting-Up-the-Cross-Compile-Environment

    The problem is, that with this setup a normal program links dynamically against: "ld-musl-mips-sf.so.1" but the onion has "ld-uClibc.so.0" installed. I know that a static linked program works, but in the end I don't want to get hello World to work ;-)

    $ readelf -l helloWorld
    Elf file type is EXEC (Executable file)
    Entry point 0x400600
    There are 9 program headers, starting at offset 52
    Program Headers:
      Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
      PHDR           0x000034 0x00400034 0x00400034 0x00120 0x00120 R E 0x4
      INTERP         0x000154 0x00400154 0x00400154 0x0001a 0x0001a R   0x1
          [Requesting program interpreter: /lib/ld-musl-mips-sf.so.1]
      REGINFO        0x000170 0x00400170 0x00400170 0x00018 0x00018 R   0x4
      LOAD           0x000000 0x00400000 0x00400000 0x008f0 0x008f0 R E 0x10000
      LOAD           0x000fdc 0x00410fdc 0x00410fdc 0x00050 0x00074 RW  0x10000
      DYNAMIC        0x000188 0x00400188 0x00400188 0x00110 0x00110 RWE 0x4
      GNU_STACK      0x000000 0x00000000 0x00000000 0x00000 0x00000 RW  0x10
      GNU_RELRO      0x000fdc 0x00410fdc 0x00410fdc 0x00024 0x00024 R   0x1
      NULL           0x000000 0x00000000 0x00000000 0x00000 0x00000     0x4
     Section to Segment mapping:
      Segment Sections...
       01     .interp 
       02     .reginfo 
       03     .interp .reginfo .dynamic .hash .dynsym .dynstr .gnu.version .gnu.version_r .rel.plt .init .text .MIPS.stubs .fini .rodata .eh_frame .plt 
       04     .ctors .dtors .got.plt .rld_map .got .sdata .bss 
       05     .dynamic 
       07     .ctors .dtors .got.plt 

    After reading "https://community.onion.io/topic/9/how-to-install-gcc" I believe that this dynamic link is what caused the problem for Kit Bishop:

    root@Omega:~# ls -l
    -rwxrwxrwx    1 root     root          8994 Nov 18 12:26 testc
    root@Omega:~# ./testc
    -ash: ./testc: not found

    (Sry but I can't find a link to his post option :-()

    I see that the tool chain uses the wrong stdLib, that path name of the tool chain looks like: "...toolchain-mips_34kc_gcc-5.2.0_musl-1.1.11/

    I have the kernel settings on "Preferred standard C++ library (uClibc++) --->" but the other uClibc settings I can't activate because mips is set and I don't know where openwrt configures the tool chain std lib.

    So the question is: How to configure the openwrt setup to compile programs for the existing onion os? How to use uClibc instead of musl?

    Just for completness here are my helloWorld Makefile:

    # build helloworld executable when user executes "make"
    helloWorld: helloWorld.o
    	@echo "\n\n----------------------\n\n"
    	@echo "CXX: $(CXX)"
    	@echo "CFLAGS: $(CFLAGS)"
    	@echo "LDFLAGS: $(LDFLAGS)"
    	@echo "LIB: $(LIB)"
    	$(CXX) $(LDFLAGS) helloWorld.o -o helloWorld
    helloWorld.o: helloWorld.c
    	$(CXX) $(CXXFLAGS) -c helloWorld.c
    all: helloWorld
    	rm *.o helloWorld

    and the openWRT project Makefile:

    include $(TOPDIR)/rules.mk
    # This specifies the directory where we're going to build the program.  
    # The root build directory, $(BUILD_DIR), is by default the build_mipsel 
    # directory in your OpenWrt SDK directory
    include $(INCLUDE_DIR)/package.mk 
    define Package/helloWorld
    define Package/helloWorld/description
    	Hello World
    # Specify what needs to be done to prepare for building the package.
    # In our case, we need to copy the source files to the build directory.
    # This is NOT the default.  The default uses the PKG_SOURCE_URL and the
    # PKG_SOURCE which is not defined here to download the source from the web.
    # In order to just build a simple program that we have just written, it is
    # much easier to do it this way.
    define Build/Prepare
    	mkdir -p $(PKG_BUILD_DIR)
    	$(CP) /home/apit/projects/openwrt/helloWorld/* $(PKG_BUILD_DIR)/
    MAKE_OPTS:= \
        ARCH="$(LINUX_KARCH)" \
    define Package/helloWorld/Compile
    	cd $(PKG_BUILD_DIR)
    	#$(MAKE) -C $(PKG_BUILD_DIR) \
    	#	CXXFLAGS="$(TARGET_CFLAGS) $(EXTRA_CPPFLAGS) -nostdinc++" \
    	#	CROSS="$(TARGET_CROSS)" \
    	#	ARCH="$(ARCH)" \
    	#	$(1);
    define Package/helloWorld/install
    	$(INSTALL_DIR) $(1)/bin/
    	$(INSTALL_BIN) $(PKG_BUILD_DIR)/helloWorld $(1)/bin/helloWorld
    $(eval $(call BuildPackage,helloWorld)) 

    Of course alot of that lines are from openwrt hello world tutorials or from onion omega projects. I also remove some lines, so if it not compile its because I erased to much :-) ... I tried to keep it clean here... sry.

    PS: thanks alot for your help!

    PSS: I found this statistic: http://www.etalabs.net/compare_libcs.html
    Is it possible that compiling the os with musl is faster and smaller then with uClibc?

  • Hi Alexander,

    Below are in a few steps my way to success in building my cross platform apps:

    1. I run a CentOS 6.5 linux box (32bit)

    2. download the 32bits mips toolchain for Onion to /usr from this link https://s3-us-west-2.amazonaws.com/onion-downloads/openwrt/OpenWrt-Toolchain-ar71xx-generic_gcc-4.8-linaro_uClibc-

    3. unpack this toolchain using:
      cd /usr
      tar -xvf OpenWrt-Toolchain-ar71xx-generic_gcc-4.8-linaro_uClibc-

    4. move the extracted dir to a shorter name:
      mv OpenWrt-Toolchain-ar71xx-generic_gcc-4.8-linaro_uClibc- onion

    5. in your app directory create Makefile.onion with following contents:

    CC = /usr/onion/toolchain-mips_34kc_gcc-4.8-linaro_uClibc-
    LD = /usr/onion/toolchain-mips_34kc_gcc-4.8-linaro_uClibc-
    CFLAGS = -I /usr/onion/toolchain-mips_34kc_gcc-4.8-linaro_uClibc-
    LDFLAGS = -L /usr/onion/toolchain-mips_34kc_gcc-4.8-linaro_uClibc-

    EXEC= hello.onion

    THREADLIB = -lpthread
    RTLIB = -lrt
    DLLIB = -ldl

    IFLAGS = -I..

    all: clean $(EXEC)


    $(EXEC): $(OBJS)
    $(CC) -o $(EXEC) $(OBJS) $(LDFLAGS)

    rm -rf $(OBJS) $(EXEC)

    $(CC) -c -o $@ $(DEBUG) $(CFLAGS) $(IFLAGS) $<

    $(CC) -c $(CFLAGS) $(DEBUG) $< -o $@ $(IFLAGS)

    hello.o: ../hello.cpp

    1. make your hello app: make -f Makefile.onion

    2. copy hello.onion to Onion and give it exec rights

  • Thank you for your suggestion,

    I know that a lot of people just use the SDK to compile programs, but I want to get the openWRT package CrossCompile environment to work. I want to create packages that I can install on the onion (.ipk packages if I'm remember correctly). I don't know how to get the toolchain you suggested into the openWRT environment, so if you have an Idea how to do that, it would be perfect <3

    I mean someone ;-) compiles their packages with the correct toolchain/setup and provide this packages to us, don't they?

  • Hi @Alexander-Pitzer, In the make menuconfig system, go into Advanced configuration options (for developers) > Toolchain Options > C Library implementation. You just need to switch from musl to uClibc.

  • @Boken-Lin


    Yeah I tried this before, too. But there is no option for uClibc....
    May it be possible to poste a .conf with the right kernel settings?

    Because the dependency for uClibc contains : .... && !mips [=y] && ....
    and mips is set by: .... || TARGET_ar71xx_generic [=y] && <choice> && TARGET_ar71xx [=y] || ....


  • @Alexander-Pitzer that's quite strange. Where did you get the buildroot files?

  • @Boken-Lin


    I believe I found the problem, in the current version of openWRT uClibc is available if broken packages are allowed. RIght now I compile the toolchain: toolchain-mips_34kc_gcc-5.2.0_uClibc-1.0.9
    They switched to uClibc-ng-1.0.9

    8e493a4b7c0749bf65cfd545ca18b384bc1d4f32    12/03/2015 03:47 PM
    uClibc-ng: update to 1.0.9
    Update to 1.0.9, switch to XZ tarball.
    Signed-off-by: Waldemar Brodkorb <wbx@uclibc-ng.org>
    git-svn-id: svn://svn.openwrt.org/openwrt/trunk@47715 3c298f89-4303-0410-b956-a3cf2f4a3e73
    b70a36d1d969f5aa9ad081c9a395633793582a5a  11/02/2015 07:12 PM
    uclibc: remove version 0.9.33
    Latest uClibc-ng is now the only supported option
    Signed-off-by: Felix Fietkau <nbd@openwrt.org>
    git-svn-id: svn://svn.openwrt.org/openwrt/trunk@47357 3c298f89-4303-0410-b956-a3cf2f4a3e73

  • @Alexander-Pitzer Oh i see. Were you using a version of the buildroot that did not include uClibc?

  • Sry I updated my post... so yes uClibc was kicked and uClibc-ng is in but only as broken package

  • @Alexander-Pitzer Bummer. I guess the only way to do it right now is to check out f14d574f3082577a4489f778d0c8cf7117d33cee commit, and come back to the new one once the broken package has been fixed...

  • @Boken-Lin I try to compile the new version and will play around with the toolchain a bit. If nothing helps I will go back. Thank's alot for your help. The Onion Omega is a great product and I really like to play around with it alot!

  • @Alexander-Pitzer I'm glad you like it :) If there's anything else we can help you with just let us know!

Log in to reply

Looks like your connection to Onion Community was lost, please wait while we try to reconnect.