Posts made by Lazar Demin

Today, we’re excited to introduce the Omega2 Eval Boards!

The new Omega2 Eval Boards are built to streamline your experience with the Omega2 IoT modules – from the very first prototype to a final, production-ready device. By bringing nearly all of the Omega2’s capabilities together on a single, compact board, these Eval Boards make it easier than ever to prototype, develop, and ship your connected projects.

For you, this means less time spent juggling different boards and expansions, and more freedom to focus on what matters most: bringing your ideas to life. By streamlining the entire process, the Eval Boards help you stay motivated, creative, and productive—whether you’re a seasoned engineer or a first-time builder.

Available in two variants – one with the classic through-hole Omega2+ module and another featuring the surface-mount Omega2S+ – these Eval Boards offer immediate, hands-on access to all of the Omega2’s I/O pins, an integrated Ethernet port for easy networking, USB-C for power and serial command line access, a MicroSD slot for expanded storage, and more. This combination of features eliminates the need for multiple add-ons and expansions. In particular, the Omega2S Eval Board delivers the expanded functionality of the Omega2S at a more accessible price than any previous solution.

We’ve also introduced new firmware based on OpenWRT 23.05, providing a modern, lightweight foundation that’s easier to customize, and a new documentation site to go along with it. With faster builds and less time spent on the getting the basics working, you’ll enjoy a smoother, more direct path to getting your ideas running in the real world.

Learn more and get your Omega2 Eval Board at Crowd Supply

@mayur_ingle said in Firmware Boot Failure on Onion Omega2+: Steady Orange LED, Stuck Bootloader Mode and Missing Hostname:

Initially, it performed as expected, but after 14 days, the device entered a stuck mode, similar to previous occurrences.

The fact that it now takes 14 days instead of 8-10 days is strange. If the cause of modules getting stuck was indeed too many writes to flash, then I would expect the problem to be resolved. If this was not the cause, I would expect the time it takes to get stuck to be the same as before. It's strange that it changed by a few days.

From a high-level, what else does the script do? Are there any other interactions with the filesystem?
How often do you move the files from /tmp to the flash storage?

---

@mayur_ingle said in Firmware Boot Failure on Onion Omega2+: Steady Orange LED, Stuck Bootloader Mode and Missing Hostname:

The script is designed to automatically reboot and restart the system if Python is detected as not running,

How are you starting the python program in the first place? And is there a specific reason why you're rebooting the whole device if the python program stops running?

A more straight forward solution would be to run the python program as a service: the system will take care of restarting it if it stops executing, without the need for a full reboot and downtime.
This is the recommended approach.

See this blog post for more information on running a program as service: https://onion.io/2bt-custom-initd-service/

---

@mayur_ingle said in Firmware Boot Failure on Onion Omega2+: Steady Orange LED, Stuck Bootloader Mode and Missing Hostname:

Do you have any further suggestions for recovering the stuck devices, given that the bootloader menu doesn’t appear to be accessible?

This is a very strange situation. The bootloader is in a completely separate partition, I'm not sure how anything done in linux can impact the bootloader partition. Especially since the bootloader partition is set to read-only from Linux...

Suggestions from my previous post:

For the stuck devices your next step should be trying them on a standard Dock from Onion, and using a simple terminal program like screen, minicom, or putty to try to activate the bootloader menu.
Otherwise, these 3 devices might be write-offs. You can try to recover them by using an external device to rewrite the flash but we (Onion) don't recommend this procedure as a lot can go wrong.

---

@mayur_ingle said in Firmware Boot Failure on Onion Omega2+: Steady Orange LED, Stuck Bootloader Mode and Missing Hostname:

Could there be an underlying hardware issue contributing to this behavior?

I find this highly unlikely. Otherwise this issue would be seen on all Omega devices eventually.
Since other users have seen a similar issue and successfully resolved it by moving the writes to /tmp, I would venture a guess that it has something to do with how the compressed filesystem reacts to very frequent writes to files.

There's one more avenue you could explore. Try running the same program (but as a service and all file writes going to /tmp, and your reboot script removed) on the new beta firmware.
The beta firmware is based on kernel 5.15, there may be updates to how the kernel interacts with the flash that could resolve the issue you're seeing.

More info and installation instructions for the beta firmware found here: https://onion.io/embracing-the-future-new-omega2-beta-firmware-and-documentation-site/

Let me know how it goes!

Cool! Thanks for the heads up @luz

I've made a github issue to track this update: https://github.com/OnionIoT/OpenWRT-Packages/issues/106

For now the plan is to make this updated driver available in the new 23.05 firmware.

Does anybody have a specific need to have this available in the older v0.3.4 firmware??

@mayur_ingle ok, glad to hear you're now seeing the expected behaviour with the working devices.

The situation is a little unusual with the stuck devices. I didn't expect the bootloader to be impacted by the file system issue.

I agree with what @crispyoz said above:

If you are using an Omega2+ (through hole) my next step would be to insert it into a standard dock and view the boot process using minicom or some other terminal software. I looked at DockLight but haven't used it, but a raw terminal would remove any potential issues of handshaking or such causing the issue.

For the stuck devices your next step should be trying them on a standard Dock from Onion, and using a simple terminal program like screen, minicom, or putty to try to activate the bootloader menu.

Otherwise, these 3 devices might be write-offs. You can try to recover them by using an external device to rewrite the flash but we (Onion) don't recommend this procedure as a lot can go wrong.

@mayur_ingle said in Firmware Boot Failure on Onion Omega2+: Steady Orange LED, Stuck Bootloader Mode and Missing Hostname:

a) On a working device:
I followed the recommended steps: powering on while holding the FW_RST pin (GPIO38) active.
Observation: This process erases the existing firmware on the working device, resets it, and allows me to re-upload the firmware. After this, folders become accessible, and the hostname is visible.
I did not see a bootloader menu during this process; it directly erased the firmware and enabled reconfiguration. In short working fine

This is not expected.
Can you elaborate on your observations? What do you mean by it erases the existing firmware on the device? What were the steps you had to do to make this happen? Can you post a log of the terminal?

Expected behaviour

If you have Omega2 devices manufactured in the last ~7 years, you should see a bootloader menu if the device is powered on with the FW_RST active:

You then need to select an option from the menu. See the Firmware Flashing With Web Recovery Mode docs article for the full process.

---

@mayur_ingle said in Firmware Boot Failure on Onion Omega2+: Steady Orange LED, Stuck Bootloader Mode and Missing Hostname:

Following your suggestion, I updated the file writes to /tmp and the device is under observation for 8 to 10 days (as my old device is got stucked after 8 days).

Great! I suspect this will resolve the issue. Let us know how it goes!

@mayur_ingle this is a great bug report, very detailed!

Since you're using the through-hole Omega2+, I agree with @crispyoz 's suggestion to try out your stuck devices on a standard dock. Just to rule out any hardware issues.

IMO the issue is more likely to be file system corruption than RAM corruption

Avoiding File System Corruption

@mayur_ingle said in Firmware Boot Failure on Onion Omega2+: Steady Orange LED, Stuck Bootloader Mode and Missing Hostname:

The main_app.py script captures Modbus data packets (32 packets of 105 bytes each) and appends them to an Excel file every 2 seconds.

I responded to your colleague on a GitHub Issue but I will post this here for visibility:

A few other users have reported file system instability when programs are running that frequently write to the flash storage. To get around this, we recommend moving any file writes to the /tmp directory (as this is actually on the RAM, not the flash).

In this case, data that should persist indefinitely should be copied over from /tmp to the flash filesystem (anything else on /) at some longer interval, perhaps daily. Cron is solid tool for this copy job.

Recovering Stuck Devices

How many stuck devices do you currently have?
I'd like to confirm if the bootloader can be accessed on a stuck device.

On a working device, the bootloader menu can be enabled by powering on the device while holding the FW_RST pin (GPIO38) active. This reset pin is active-high, and this is the pin used by the reset button on the Omega2 Docks.
Keep in mind pressing the enter or space keys will not activate the bootloader menu.

Please try this first on a working device, and then try it on a "stuck" device. Report back how it goes.

@crispyoz Your PoE suggestion is interesting! Can you give us a little more detail on why, when, and how PoE is helpful?

This goes for all users!

Who knows? Maybe we make a variant with onboard PoE

Yep!
Our new Omega2 Eval Boards represent a new starting point for working with the Onion Omega2 modules. Designed to lower the friction of working with the Omega2, they’re indispensable for evaluation, early prototyping, and development. They also serve as a template to build your own custom board and can even be used as SBCs for your applications.

We’d love for you to give our campaign page a look and consider backing our Omega2 Eval Boards!

hi @mauriziomeucci, sorry to hear about the frustration caused by not having the avahi daemon.

To quote @crispyoz:

The term "Beta" indicating that it is a test release inviting feedback, which would include which packages users may prefer to be included.

The 23.05 firmware is still in beta and not yet our stable firmware. We want to give users a chance to try it out before rolling it out as the default firmware in the factory.

If you have specific improvement suggestions (or bugs to report), you can create an Issue in the OnionIoT/OpenWRT-Packages GitHub Repo. There are issue templates you can use for Improvement Suggestions/Feature Requests and Bug Reports.

You can also try building your own version of the 23.05 firmware that includes the avahi daemon. This is now very easy and takes minutes instead of hours. See the How to Build Custom Firmware article in the new documentation for the procedure.

Let me know how it goes!

---

@luz what's a quick way to test if IPv6 and link local is enabled in the new fw? I'm not too familiar with IPv6 - I need to study

What’s new?

Two big items!

1. New Documentation site for the new firmware is available
2. Firmware based on OpenWRT release 23.05.3 is available!

New Documentation Site

Documentation for the “new” firmware based on OpenWRT 23.05 is now live at: https://documentation.onioniot.com/

This will be the main documentation site going forward!

Just like we started with a clean slate for the firmware, this is a clean slate for documentation. Our goal was to start from scratch, organize the documentation in a way that makes it useful to readers, and not carry over every single item from the existing legacy documentation.

We used Docusaurus - a modern static website generator - as the underlying platform for the documentation site instead of a homebrew solution. It is easy to maintain and deploy, very customizable, and looks quite good in our opinion.

The documentation is open source and available on GitHub: https://github.com/OnionIoT/documentation

Work on the documentation is still in progress - so expect to see updates.

Our goal is to make this documentation as useful as possible for our users. To make that happen we’ll need your help! We’ve added comment sections to each article - if anything is missing, confusing, or needs attention, login with Github and drop a comment right in the article.

You can also give us longer feedback by creating an issue in the OnionIoT/documentation GitHub repo. We also accept PRs for direct contributions!

We’re happy to hear your thoughts on what information you need to successfully work with the Omega and update the documentation accordingly!

What about documentation for the v0.3.4 firmware??

The existing documentation at https://docs.onion.io/ will remain available and focused on the v0.3.4 (and earlier) firmware. We will make updates there as needed. But expect it to remain largely the same.

We will call this the legacy documentation site going forward.

New Firmware based on OpenWRT 23.05

A new release of Onion customized firmware is now available!

Highlights include:

• Migrated to the OpenWRT 23.05.3 release
• Running Linux kernel 5.15
• Run-time loadable Device Tree Overlays are fully supported - easily extend the hardware capabilities of the device within Linux
  • No longer have to build firmware from scratch to support additional hardware!
• Added support for more hardware interfaces
  • Hardware PWM - learn more here.
  • p44-ledchain to control WS281x LEDs (Neopixels) - learn more here.
  • One Wire support (using DT Overlay) - learn more here.
  • Better SPI support - using DT Overlay for software-based SPI bus and Python spidev module added for both software and hardware SPI bus control - learn more here and here.
  • Quectel EC25 LTE modem support (used on the Omega2 LTE) - learn more here.
• Fixed the GPIO numbering - GPIO numbers in Linux match the Omega2 pinout diagrams
• Supports NodeJS v18 - as far as we know, modern versions of Node are not supported on any other MIPS platform - learn more here.

How can I get and use this firmware?

The firmware is available in the Onion software repository: http://repo.onioniot.com/omega2/images/openwrt-23.05/

See the Installing the New Firmware article on the new documentation site for instructions on how to take this firmware for a spin

️ The New Firmware works differently from previous versions ️
Try it out on a small scale before rolling it out to production! For a detailed list of changes to the firmware, see the What’s New article on the new documentation site.

Is it easy to build my own version of this firmware?

That’s a great question!

We’ve heard from many users that making customizations to the Onion firmware is hard, confusing, and time-consuming. We’ve put in work to update and simplify this process - it now takes minutes instead of hours! Spend less time setting up the environment and building everything from scratch, and use that time to focus on your application!

See the full break-down of how we build customized firmware in the new documentation.

In a nutshell:

1. All customizations are encapsulated as
   • Patches to the OpenWRT Build System
   • Software Packages
2. We use the Build System to create customized versions of the OpenWRT SDK and Image Builder
3. Onion packages are compiled using the OpenWRT SDK - learn more here.
4. Firmware (featuring Onion packages) is built using the OpenWRT Image Builder - learn more here.

To make your own customized packages and firmware, we suggest doing steps 3 and 4 with some modifications for your own purposes!

How can I report bugs or give feedback?

We value your input and are eager to hear about your experiences with the firmware. Feel free to post in this thread - I’ll be around to chat!

If you have specific improvement suggestions (or bugs to report), you can create an Issue in the OnionIoT/OpenWRT-Packages GitHub Repo. There are issue templates you can use for Improvement Suggestions/Feature Requests and Bug Reports.

What happened to the earlier 22.03 beta firmware?

We continued working on it! I neglected to update the community on our progress - that’s my bad.

After the previous update on the 22.03 firmware we followed through on the next steps from that post:

• Upgraded to the OpenWRT 22.03.5 release
• Add support for:
  • Hardware PWM
  • p44-ledchain to control WS281x LEDs (Neopixels)
  • Device Tree overlays loadable during run time
  • Added one-wire support
  • Added Python spidev module for interacting with the SPI bus
  • Added software SPI support

This firmware will remain available at: http://repo.onioniot.com/omega2/images/openwrt-22.03/

But our future development efforts will be on the OpenWRT 23.05-based firmware!

What’s Next?

You tell us what you’d like to see and we’ll do our best to make it happen

We currently are working on the following:

• Some new hardware is on the horizon - more news on that soon!
• We’re going to continue working on the documentation and firmware based on your feedback
• At some point later this year or early next year, new Omega devices will ship with the OpenWRT 23.05-based firmware - we will give advance notice before this happens!

---

EDIT Oct 18, 2024: fixed issue with broken links in post
EDIT Nov 8, 2024: added warning to try out new firmware on small scale first

@crispyoz this is a great example on creating a new user for ubus use! You beat us to the punch! Could you make a new FAQ?

@crispyoz said in OnionOS user:

I haven't used the OnionOS stuff for a few years but as I recall it uses the the ubus acls

That's right! We use the root user though!

@mrahul said in OM2+ getting stuck in boot stage with custom firmware using b255:

Should we see the same behavior if we use the binary downloaded from the Onion FW Repo?

No, my previous comment does not apply to firmware in the Onion firmware repo - these firmware images are already compiled and should contain everything needed.

So devices with mac addresses that start with 40:A3:6B should boot and operate properly when flashed with any firmware from the Onion firmware repo.

Devices with mac address addresses starting with 88:1E:59 will boot and operate correctly when flashed with firmware b255 and higher.

I've confirmed this with devices I have on hand
Device with 40:A3:6B mac address running firmware b254:

Device with 40:A3:6B mac address running firmware b259:

Device with 88:1E:59 mac address running firmware b256:

---

@mrahul said in OM2+ getting stuck in boot stage with custom firmware using b255:

However, we only use two methods, i.e. through ethernet or syupgrade to upgrade or re-flash a device on bricking. How different are these two processes?

For a device that boots into linux successfully, upgrading the firmware thru sysupgrade and thru ethernet with the bootloader will have the same effect. As long as sysupgrade is run with the -n option to overwrite the existing configuration and filesystem.

For a device that cannot boot into Linux, we recommend reflashing the firmware with ethernet thru the bootloader.

---

@mrahul said in OM2+ getting stuck in boot stage with custom firmware using b255:

Lately, we've started seeing the same behavior again with b255 where it gets stuck in the boot process.

Does this mean the firmware sometimes works and sometimes doesn't?
This could point to something in the hardware design impacting the boot sequence. Do you use any SPI devices? Do you follow the bootstrapping pins guidelines in your hardware?

---

My recommendation

Since you mentioned you're using the through-hole Omega2+, we can try to isolate where the issue is coming from.

I recommend doing the following:

• Remove an Omega2+ device from your custom board
• Plug it into an Onion Expansion Dock with an Ethernet Expansion
• Use the bootloader + ethernet to flash firmware b259 from the Onion firmware repo
• Observe the outcome, given my testing I expect this device should boot properly
• Plug the device back into your custom board and attempt to boot it again

Let me know how it goes!

@crispyoz Cool, I'll try to carve out some time to test drive it

Hi @Jose
If the SIM7600G LTE modem is USB enabled it should be compatible with the Omega2 devices. You will probably need to inquire with the modem manufacturer if there are any additional drivers needed for support in Linux.

If the modem supports QMI, you might be able to use existing tools to configure your modem. See this post for more details: https://openwrt.org/docs/guide-user/network/wan/wwan/ltedongle

I would also recommend seeing if the Omega2 LTE single board computer would work for your needs. For more info see: https://onion.io/omega2-lte/

@crispyoz ooo that's good to know, thanks for sharing!

Does it also support DTS fragments for overlays?
For example: https://github.com/OnionIoT/OpenWRT-Packages/blob/openwrt-23.05/onion-dt-overlay/src/sw-spi.dts

@DocHardinger Haven't tried that once specifically, but I can't think of any reason a USB 2.0 Hub won't work with the Omega. That is, as long as it's used in the design as intended by the manufacturer.

Let us know how it goes!

Hi @DocHardinger
It looks like the GL850G-OHY37 model we recommend in the hardware design guide is now EOL.

We recommend looking into the newer Genesys GL850G and GL852G lines, with the latest models being the GL850G-xxY60 and GL852G-xxY60, such as GL852G-OHY60 (QFN-28) and GL850G-HHY60 (SSOP-28).

Here is the key difference between the GL850G and GL852G lines:

We'll look into updating the hw design guide.
Hope this helps!

Thanks for finding this issue and submitting a PR @crispyoz !
This was an oversight on my end (I blame lack of sleep due to newborn)

The PR has been merged and the issue has been resolved, the OnionIoT/openwrt-imagebuilder-wrapper will now work as intended out of the box.

@mikeH happy to help!

The firmware API returning b233 as stable was due to an oversight. This has been fixed, it now returns:

{
  "version": "0.3.4",
  "url": "http://repo.onioniot.com/omega2/images/omega2p-v0.3.4-b258.bin",
  "build": 258,
  "device": "omega2+",
  "stable": true
}

Thanks for bringing this to our attention

@davidsi Looks like GTK doesn't support writing directly to the framebuffer and needs a window system, see https://discourse.gnome.org/t/use-of-gtk-on-embedded-linux-systems/8881

The Omega2 Dash only supports the framebuffer (a windowing system is probably too heavy for an embedded system)

They mention Qt in the linked thread, looks like there's good documentaiton for Qt for embedded linux: https://doc.qt.io/qt-5/embedded-linux.html