Persistence DB file not created with mosquitto mqqt broker

[DocHardinger]

@crispyoz Yes were using beta build 22.03.05 with custom build. But i already tried it with standard beta build 22.03.05 from onion. Same issues.

i will try now with the new settings in mosquitto.conf.

[crispyoz]

@DocHardinger also try not setting the persistence location, it will default to /var/lib/mosquitto/mosquitto.db

If none of these work then I start to wonder if persistence was not included in the build options, I can check on my owt 22 unit when I'm at home later.

[DocHardinger]

So i tried the following. But i didnt get any response...Or how is it working with SIGUSR2

CONFIG:

# mqtt over websocket configuration
log_type error
log_type warning
persistence true
persistence_location /etc/mosquitto/
persistence_file mos.db
autosave_on_changes false
autosave_interval 5
pid_file /etc/mosquitto/mosquitto.pid
listener 1883 0.0.0.0
protocol mqtt
allow_anonymous true
listener 9001 0.0.0.0
protocol websockets
allow_anonymous true
check_retain_source true
retain_available true

SHELL:

22353 root      1312 S    -ash
22514 mosquitt  4748 S    mosquitto -c /etc/mosquitto/mosquitto.conf
22541 root         0 IW   [kworker/u2:1-ev]
22556 root      1320 R    ps
28557 root      2664 S    {ntpd} /sbin/ujail -t 5 -n ntpd -U ntp -G ntp -C /etc/capabilities/ntpd.json -c -u -r /bin/ub
28583 ntp       1312 S    /usr/sbin/ntpd -n -N -S /usr/sbin/ntpd-hotplug -p 0.openwrt.pool.ntp.org -p 1.openwrt.pool.nt
root@E1NS-d7d3:~# kill -SIGUSR1 22541
root@E1NS-d7d3:~# kill -SIGUSR1 22514
root@E1NS-d7d3:~# kill -SIGUSR2 22514
root@E1NS-d7d3:~#

[DocHardinger]

@crispyoz i used the openwrt beta 22.03. original from onion and installed mosquitto with opkg (the ssl version) but still the same issue, i also tried without setting the directory for the db file, but no success.

[crispyoz]

@DocHardinger I tested on a stock OpenWrt 22 build, I also had an issue with the persistence. There was nothing in the logs but it was having an issue creating the database. If you run it in the console instead of as a service then you can see the error. This is mosquitto nossl version 2.0.18

Stop and disable the service:
service mosquitto stop
service mosquitto disable

Now run mosquitto in the console:

mosquitto -c /etc/mosquitto/mosquitto.conf

In my case I set the database to be stored in my data directory /etc/data

mkdir /etc/data
chown mosquitto /etc/data

This is the error I was receiving:

[DocHardinger]

i did exact the same but not even getting an error. Same configuration as in my previous post and getting:

root@E1NS-d7d3:~# service mosquitto stop
root@E1NS-d7d3:~# service mosquitto disable
root@E1NS-d7d3:~# mosquitto -c /etc/mosquitto/mosquitto.conf

^Croot@E1NS-d7d3:~# mkdir /etc/data
root@E1NS-d7d3:~# chown mosquitto /etc/data
root@E1NS-d7d3:~# mosquitto -c /etc/mosquitto/mosquitto.conf
root@E1NS-d7d3:~# mosquitto -c /etc/mosquitto/mosquitto.conf

root@ABCD-d7d3:~# mosquitto -c /etc/mosquitto/mosquitto.conf
1718314307: mosquitto version 2.0.18 starting
1718314307: Config loaded from /etc/mosquitto/mosquitto.conf.
1718314307: Opening ipv4 listen socket on port 1883.
1718314307: Opening websockets listen socket on port 9001.
1718314307: mosquitto version 2.0.18 running
1718314307: New connection from 192.168.1.175:50934 on port 1883.

NO database created, also tried your folder.
Can you post your config?

[crispyoz]

@DocHardinger

You can download my ipk and config here:
https://drive.proton.me/urls/R5XPJM90XG#LIJ04l5vml8R

This was a build from openwrt git main with only the minimum required changes to the config

I uploaded both the ssl and nonssl ipk's, I ran my tests on the ssl version.

[DocHardinger]

@crispyoz Tried this but no success because of different architecture. But what i now find out is that version 2.0.15-1 and 2.0.17 of mosquitto is working as expected. Downloaded it from original openwrt packages ( https://archive.openwrt.org/releases/packages-22.03/mipsel_24kc/packages/ ). Version 2.0.18 from same source isnt. So there seems to be a bug inside the mosquitto package.

root@E1NS-d7d3:~# mosquitto -c /etc/mosquitto/mosquitto.conf
1718349698: mosquitto version 2.0.15 starting
1718349698: Config loaded from /etc/mosquitto/mosquitto.conf.
1718349698: Opening ipv4 listen socket on port 1883.
1718349698: Opening websockets listen socket on port 9001.
1718349698: mosquitto version 2.0.15 running
1718349699: New connection from 192.168.1.175:53421 on port 1883.
1718349699: New client connected from 192.168.1.175:53421 as mqtt-explorer (p2, c1, k60, u'pc').
1718349713: Saving in-memory database to /etc/data//mosquitto.db.
1718349817: New connection from 192.168.1.1:57055 on port 1883.
1718349817: New connection from 192.168.1.1:34947 on port 1883.
1718349817: New connection from 192.168.1.1:49321 on port 1883.
1718349817: New client connected from 192.168.1.1:34947 as e1ns_bridge2 (p2, c1, k60).
1718349817: New client connected from 192.168.1.1:57055 as e1ns_automation (p2, c1, k60).
1718349817: New client connected from 192.168.1.1:49321 as e1ns_hardware (p2, c1, k60).
1718349817: Saving in-memory database to /etc/data//mosquitto.db.
1718349818: Saving in-memory database to /etc/data//mosquitto.db.
1718349818: Saving in-memory database to /etc/data//mosquitto.db.
1718349818: Saving in-memory database to /etc/data//mosquitto.db.

[crispyoz]

@DocHardinger What different architecture are you referring to? The ipk I shared was for Omega2+.

[DocHardinger]

@crispyoz yeah its strange i tried your ipk but gets an error because of architecture. When im using the ipk from the source of my last post its working fine. Were using an Omega2+ with custom build firmware.

root@E1NS-d7d3:/tmp# opkg remove mosquitto-ssl
Removing package mosquitto-ssl from root...
Command failed: Not found
Not deleting modified conffile /etc/mosquitto/mosquitto.conf.
Not deleting modified conffile /etc/config/mosquitto.
root@E1NS-d7d3:/tmp# opkg install mosquitto-ssl_2.0.18-r3_mipsel_24kc.ipk
Collected errors:
 * pkg_hash_check_unresolved: cannot find dependency libopenssl3 for mosquitto-ssl
 * pkg_hash_fetch_best_installation_candidate: Packages for mosquitto-ssl found, but incompatible with the architectures configured
 * opkg_install_cmd: Cannot install package mosquitto-ssl.
root@E1NS-d7d3:/tmp#

[crispyoz]

@DocHardinger it just wants libopenssl3 installed, I assumed you had it installed on you.r system already. You should just be able to install libopenssl3 using opkg install libopenssl3.

I also shared the non-ssl version since you don't seems to be using ssl.

[DocHardinger]

@crispyoz Yes and and some other libraries. Problem is that i need internet connection to install this or all the single packages.

Anyway we will use mosquitto 2.0.17, but im not sure how to integrate this when building the firmware. normally we use the profile file for this but then it takes 2.0.18. So how can we achieve to use 2.0.17 when building the firmware?

[crispyoz]

@DocHardinger

In your feeds.conf, you can specify the source repository and branch, here is an example of the feed I use for plan44 repo just fine the branch for the version you want and add it like so:

src-git plan44 https://github.com/plan44/plan44-feed.git;main

Then run:
scripts/feeds update -a
scripts/feeds install -a

This should add the feed for the branch you want to your make menuconfig

Another option is to make your own source feed locally, you can see the details how to do this in the developer docs here

[crispyoz]

@DocHardinger Third option is basically the same as the second option but instead of placing any source there just duplicate the existing mosquitto package directory in your local directory and modify the Makefile to change the version you want. Make will then pull down the source for the version you specify:

PKG_NAME:=mosquitto
PKG_VERSION:=2.0.18
PKG_RELEASE:=3

You'll also need to amend the hash.

You probably need to amend the name also so you can distinguish the two versions in the make menuconfig menus.

The existing package will be in <build directory>/feeds/packages/net/mosquitto

[DocHardinger]

I now just integrated the the mosquitto 2.0.17 package inside the firmware and removed the Mosquitto 2.0.18 package from the profile file. The i install the package after booting with the new firmware. So i just hope mosquitto will be fixed in the feature and i just can use the standard package again. If they didnt fix it i will go your way.

Thx for your help, its a nice experience to get answers really soon in the onion community!