Remote upgrade, keep user's data



  • I'm creating embedded device based on Onion Omega 2+. There are some python scripts to control GPIO's and some PHP and HTML as User Interface. It works well.

    Now i'm looking for solution how to make remote upgrade. User will download a file, upload it via PHP to Onion and some python script will start upgrading system. And i need to keep user files (saved gpio data, logs, passwords, settings...).

    I've tried some ways how do to it but without finish success.

    Choice 1: (rewrite application data)
    Working on some local Onion Omega. Creating my application (php, html, python, ...) in one folder (for exmaple /app-data). Then backup this folder and send it to user. On remote device script will rewrite this folder.
    This is possible but i'm worry about new installed packages and changes out of app-data. And i don't want to check all dependencies.

    Choice 2: (dd and mtd -r write)
    Working on some local Onion Omega, then make backup with dd if=/dev/mtd3 of=/mnt/backup/firmware.bin and on remote device do mtd -r write /tmp/upload/firmware.bin firmware. That works fine but i'm not able to save somewhere user's data. Is it possible to keep somewhere user's files?

    Choice 3: (rewrite /overlay)
    Working on some local Onion Omega, backup /overlay to firmware.tgz. On remote device save user files to /tmp, rewrite /overlay from firmware.tgz, copy back user files. I don't know if this way is possible. I make some tries, but with no success. After reboot Onion restore factory settings.

    Choice 4: (Compile own firmware)
    Compile my own firmware and on remote device just do sysupgrade. I've successfully compile original firmware (https://docs.onion.io/omega2-docs/cross-compiling.html), but i don't know how to add there new packages and files.

    Is there any another possibility how to do it better?
    Is any of my choice good and i'm just missing some detail?

    Radim



  • Hi @Radim-Vesely

    OpenWrt's sysupgrade probably already does a lot of what you need. For example, it has a simple mechanism to keep any files or directories across updates. Have a look at the files in /lib/upgrade/keep.d/, these list paths that must be preserved. You can simply put a file of your own there, listing the extra paths you need to be preserved (and usually, itself, to make sure it is also kept).

    Also have a look at sysupgrade's -f option, which allows you to specify a .tar.gz archive which should be applied after the upgrade. This can even be a remote url, so you can utilize that to automatically install some extra things on top of a standard update.

    So even without a custom image, you can already do a lot just with sysupgrade.

    But of course, a custom image gives you full control of the entire distribution. For anything you need to maintain for a longer time, I strongly recommend it (having done a dozen projects this way already myself). There is a learning curve, but once you've get a bit accustomed, OpenWrt is a clean and comfortable environment to maintain even complex projects.

    You can easily organize the extra packages you need in a source feed of your own (just a git repo somewhere). Copy feeds.conf.default to feeds.conf and add your feed url there. Then you can use the ./scripts/feeds utility to install (=make available for building) those packages, and once that's done, the packages from your feed will be available for choosing via make menuconfig, and including them into the firmware image at the next make.

    Creating packages is not too complicated either, once you understand what it is: a meta-makefile describing a) where to get the source, b) what the depenencies are c) how to build, d) what build results or static files need to be copied where in the target system.
    Fo simple packages you can omit a) and just include the sources into the package itself.

    Now that's only a rough summary to give an overview. It's definitely worth digging in a bit, for example about the build system in general and creating packages. There's a lot more useful info in that wiki...

    Most of my own custom builds are available with some explanatory README in my public openwrt feed. For example, p44ttngw-config builds a LoRA gateway firmware on top of LEDE 17.01.



  • Hi Luz,
    thanks for your answer and tips how to continue in my blind route :) I will look at it.

    Radim



Looks like your connection to Community was lost, please wait while we try to reconnect.