1. Home
  2. Omega Talk
  3. Persistence DB file not created with mosquitto mqqt broker

Persistence DB file not created with mosquitto mqqt broker

  • D

    We are using mosquitto-ssl V2.0.18 in OpenWRT 22.03.5. We having issues with persistence. The persistence db file is never been created.
    Permissions are set correctly in /etc/mosquitto directory (Owner is mosquitto:mosquitto). We think in Version 2.0.15 this worked fine. But its hard to downgrade to version 2.0.15 in OpenWRT to test it. There are no error messages in log.

    Steps to reproduce:

    Starting mosquitto service
    Connect to Broker with MQTT Explorer and send a retained message
    Closing connection
    Stopping mosquitto service
    Start mosquitto service
    Connect again to broker and the retained message is not published
    We are using this configuration file:

    log_type all
persistence true
autosave_on_changes false
persistence_location /etc/mosquitto/
autosave_interval 5
# mqtt configuration
listener 1883
protocol mqtt
allow_anonymous true

    Ive posted it already on Github mosquitto repo, but may be someone here hace the same problems or someone knows how to solve it. Not sure but maybe this only is an issue on omega2 hardware...

    0
  • C

    @DocHardinger retained messages can be tricky, switch on check_retain_source and retain_available.

    To cause mosquitto to write to the persistence store send the mosquitto process SIGUSR1, then wait a moment and send SIGUSR2 to get a report of subscribers and retained messages.

    To send the signals

    1. find the process id (PID) using ps -A
    2. send the signal using the command kill -SIGUSR2 <PID>
    0
  • D

    Thx for your answer i already tried with Sigusr1 but will try again with also sigusr2 maybe this gives more info. Messages are retained, i can check this by reconnect to the broker without stopping the broker,so this has been working.

    Where can i switch on check_retain_source and retain_available? In mosquitto.conf?

    The strange thing ist that exakt the same config was working in older version of openwrt/mosquitto...

    0
  • C

    @DocHardinger Are you using the Onion beta 22 or is this a custom build? I'll check on on of my devices.

    You can turn these options on by just adding them to the mosquitto.conf just add them to the bottom of the file and set to true.

    0 D 1 Reply
  • D

    @crispyoz Yes were using beta build 22.03.05 with custom build. But i already tried it with standard beta build 22.03.05 from onion. Same issues.

    i will try now with the new settings in mosquitto.conf.

    0
  • C

    @DocHardinger also try not setting the persistence location, it will default to /var/lib/mosquitto/mosquitto.db

    If none of these work then I start to wonder if persistence was not included in the build options, I can check on my owt 22 unit when I'm at home later.

    0 D 1 Reply
  • D

    So i tried the following. But i didnt get any response...Or how is it working with SIGUSR2

    CONFIG:

    # mqtt over websocket configuration
log_type error
log_type warning
persistence true
persistence_location /etc/mosquitto/
persistence_file mos.db
autosave_on_changes false
autosave_interval 5
pid_file /etc/mosquitto/mosquitto.pid
listener 1883 0.0.0.0
protocol mqtt
allow_anonymous true
listener 9001 0.0.0.0
protocol websockets
allow_anonymous true
check_retain_source true
retain_available true

    SHELL:

    22353 root      1312 S    -ash
22514 mosquitt  4748 S    mosquitto -c /etc/mosquitto/mosquitto.conf
22541 root         0 IW   [kworker/u2:1-ev]
22556 root      1320 R    ps
28557 root      2664 S    {ntpd} /sbin/ujail -t 5 -n ntpd -U ntp -G ntp -C /etc/capabilities/ntpd.json -c -u -r /bin/ub
28583 ntp       1312 S    /usr/sbin/ntpd -n -N -S /usr/sbin/ntpd-hotplug -p 0.openwrt.pool.ntp.org -p 1.openwrt.pool.nt
root@E1NS-d7d3:~# kill -SIGUSR1 22541
root@E1NS-d7d3:~# kill -SIGUSR1 22514
root@E1NS-d7d3:~# kill -SIGUSR2 22514
root@E1NS-d7d3:~#
    0
  • D

    @crispyoz i used the openwrt beta 22.03. original from onion and installed mosquitto with opkg (the ssl version) but still the same issue, i also tried without setting the directory for the db file, but no success.

    0
  • C

    @DocHardinger I tested on a stock OpenWrt 22 build, I also had an issue with the persistence. There was nothing in the logs but it was having an issue creating the database. If you run it in the console instead of as a service then you can see the error. This is mosquitto nossl version 2.0.18

    Stop and disable the service:
    service mosquitto stop
    service mosquitto disable

    Now run mosquitto in the console:

    mosquitto -c /etc/mosquitto/mosquitto.conf

    In my case I set the database to be stored in my data directory /etc/data

    mkdir /etc/data
    chown mosquitto /etc/data

    b6bb660a-b260-476b-9751-8b6db58e8eca-image.png

    This is the error I was receiving:

    69d61d74-5363-4db3-bfd6-159170d4c84d-image.png

    0
  • D

    i did exact the same but not even getting an error. Same configuration as in my previous post and getting:

    
root@E1NS-d7d3:~# service mosquitto stop
root@E1NS-d7d3:~# service mosquitto disable
root@E1NS-d7d3:~# mosquitto -c /etc/mosquitto/mosquitto.conf

^Croot@E1NS-d7d3:~# mkdir /etc/data
root@E1NS-d7d3:~# chown mosquitto /etc/data
root@E1NS-d7d3:~# mosquitto -c /etc/mosquitto/mosquitto.conf
root@E1NS-d7d3:~# mosquitto -c /etc/mosquitto/mosquitto.conf

root@ABCD-d7d3:~# mosquitto -c /etc/mosquitto/mosquitto.conf
1718314307: mosquitto version 2.0.18 starting
1718314307: Config loaded from /etc/mosquitto/mosquitto.conf.
1718314307: Opening ipv4 listen socket on port 1883.
1718314307: Opening websockets listen socket on port 9001.
1718314307: mosquitto version 2.0.18 running
1718314307: New connection from 192.168.1.175:50934 on port 1883.

    NO database created, also tried your folder.
    Can you post your config?

    0
  • C

    @DocHardinger

    You can download my ipk and config here:
    https://drive.proton.me/urls/R5XPJM90XG#LIJ04l5vml8R

    This was a build from openwrt git main with only the minimum required changes to the config

    I uploaded both the ssl and nonssl ipk's, I ran my tests on the ssl version.

    0 D 1 Reply
  • D

    @crispyoz Tried this but no success because of different architecture. But what i now find out is that version 2.0.15-1 and 2.0.17 of mosquitto is working as expected. Downloaded it from original openwrt packages ( https://archive.openwrt.org/releases/packages-22.03/mipsel_24kc/packages/ ). Version 2.0.18 from same source isnt. So there seems to be a bug inside the mosquitto package.

    root@E1NS-d7d3:~# mosquitto -c /etc/mosquitto/mosquitto.conf
1718349698: mosquitto version 2.0.15 starting
1718349698: Config loaded from /etc/mosquitto/mosquitto.conf.
1718349698: Opening ipv4 listen socket on port 1883.
1718349698: Opening websockets listen socket on port 9001.
1718349698: mosquitto version 2.0.15 running
1718349699: New connection from 192.168.1.175:53421 on port 1883.
1718349699: New client connected from 192.168.1.175:53421 as mqtt-explorer (p2, c1, k60, u'pc').
1718349713: Saving in-memory database to /etc/data//mosquitto.db.
1718349817: New connection from 192.168.1.1:57055 on port 1883.
1718349817: New connection from 192.168.1.1:34947 on port 1883.
1718349817: New connection from 192.168.1.1:49321 on port 1883.
1718349817: New client connected from 192.168.1.1:34947 as e1ns_bridge2 (p2, c1, k60).
1718349817: New client connected from 192.168.1.1:57055 as e1ns_automation (p2, c1, k60).
1718349817: New client connected from 192.168.1.1:49321 as e1ns_hardware (p2, c1, k60).
1718349817: Saving in-memory database to /etc/data//mosquitto.db.
1718349818: Saving in-memory database to /etc/data//mosquitto.db.
1718349818: Saving in-memory database to /etc/data//mosquitto.db.
1718349818: Saving in-memory database to /etc/data//mosquitto.db.
    0
  • C

    @DocHardinger What different architecture are you referring to? The ipk I shared was for Omega2+.

    0 D 1 Reply
  • D

    @crispyoz yeah its strange i tried your ipk but gets an error because of architecture. When im using the ipk from the source of my last post its working fine. Were using an Omega2+ with custom build firmware.

    root@E1NS-d7d3:/tmp# opkg remove mosquitto-ssl
Removing package mosquitto-ssl from root...
Command failed: Not found
Not deleting modified conffile /etc/mosquitto/mosquitto.conf.
Not deleting modified conffile /etc/config/mosquitto.
root@E1NS-d7d3:/tmp# opkg install mosquitto-ssl_2.0.18-r3_mipsel_24kc.ipk
Collected errors:
 * pkg_hash_check_unresolved: cannot find dependency libopenssl3 for mosquitto-ssl
 * pkg_hash_fetch_best_installation_candidate: Packages for mosquitto-ssl found, but incompatible with the architectures configured
 * opkg_install_cmd: Cannot install package mosquitto-ssl.
root@E1NS-d7d3:/tmp#
    0
  • C

    @DocHardinger it just wants libopenssl3 installed, I assumed you had it installed on you.r system already. You should just be able to install libopenssl3 using opkg install libopenssl3.

    I also shared the non-ssl version since you don't seems to be using ssl.

    0 D 1 Reply
  • D

    @crispyoz Yes and and some other libraries. Problem is that i need internet connection to install this or all the single packages.

    Anyway we will use mosquitto 2.0.17, but im not sure how to integrate this when building the firmware. normally we use the profile file for this but then it takes 2.0.18. So how can we achieve to use 2.0.17 when building the firmware?

    0
  • C

    @DocHardinger

    In your feeds.conf, you can specify the source repository and branch, here is an example of the feed I use for plan44 repo just fine the branch for the version you want and add it like so:

    src-git plan44 https://github.com/plan44/plan44-feed.git;main

    Then run:
    scripts/feeds update -a
    scripts/feeds install -a

    This should add the feed for the branch you want to your make menuconfig

    Another option is to make your own source feed locally, you can see the details how to do this in the developer docs here

    0
  • C

    @DocHardinger Third option is basically the same as the second option but instead of placing any source there just duplicate the existing mosquitto package directory in your local directory and modify the Makefile to change the version you want. Make will then pull down the source for the version you specify:

    PKG_NAME:=mosquitto
    PKG_VERSION:=2.0.18
    PKG_RELEASE:=3

    You'll also need to amend the hash.

    You probably need to amend the name also so you can distinguish the two versions in the make menuconfig menus.

    The existing package will be in <build directory>/feeds/packages/net/mosquitto

    0
  • D

    I now just integrated the the mosquitto 2.0.17 package inside the firmware and removed the Mosquitto 2.0.18 package from the profile file. The i install the package after booting with the new firmware. So i just hope mosquitto will be fixed in the feature and i just can use the standard package again. If they didnt fix it i will go your way.

    Thx for your help, its a nice experience to get answers really soon in the onion community!

    0
mqtt 17 mosquitto 12 database 7 permissions 3
Log in to reply
 

Looks like your connection to Community was lost, please wait while we try to reconnect.