Any plan to upgrade to recent IDF?

[mgliewe]

Is there any plan to upgrade to a more recent idf?

I would be willing to volunteer, but it might involve some major changes in the codebase and the build system.

Topics involving..

• moving from make to cmake
• get ridd of the OS patches, bound to a specific idf release
• adopt to API changes from IDF 3.xx to 4.xx
• maybe get ridd of the custon ld scripts? are these portable?
• ensure portability for the various supported esp boards

sugestions?

[the0ne]

That sounds reasonable. Did you make progress yet?

[a2800276]

I would be interested in helping out as well.
@mgliewe were you just feeling the water or have you started kicking the tires to see how feasible the migration would be?

[mgliewe]

due to lack of time, till now I basically just browsed the code to assess the effort.

A first naive attempt, just tweak the makefiles until you get any binary, failed marvelously, but i didn't expect otherwise and didn't put much time into that though.

I'ld guess the first point on my list would be the starting point. On working on the build files one should try to reduce the cross-dependencies between the components...
This could be done on the legacy IDF?

[a2800276]

This could be done on the legacy IDF?

Wasn't the point of this issue to discuss migration to a current IDF? Or are you asking if it makes sense to first upgrade to the most recent Makefile based IDF?

[mgliewe]

After even more examinations and some very annoying hacking sessions on dropbear (trying to work on #404, porting unixish processes and signals to a thread model is just no fun; got it somewhat working, but with ugly hacks and the prospect to port 'scp -t' and implement a sftp server from scratch) i decided to drop the topic and chose another path.

My reasons, all of them very subjective, mostly a matter of taste, and not at all ment to be taken as an insult or devaluation of this work, i still like it and found it very inspiring:

• the code base is to much behind the current idf development, and very specific to one specific idf version (which was the topic after all)
• i don't see the point in having >=5 different crypto modules (esp-idf, dropbear, libssh, curl, openssh, bootloader), bloating the firmware and being hard to maintain
• the codebase is way to much esp-idf centristic and hard to emulate on a host workstation
• i don't really like the idea of having just one lua context running, and accessing it from several threads. I'ld rather have a command & control context, a seperate httpd context and means to install some kind of 'drivers' for hardware control. And some tools to manipulate them (creat/destroy/attach/you name it)
• the project is just hard to make portable to different platforms (i said that before, didn't i? :) )
• no async io support; its lacking a proper eventing system; nowerdays i would expect promises, or at least node.js style callbacks; startup the wifi, then do that, if it failed blink the led....

Right now I'm working on a port of cyclone tcp (https://www.oryx-embedded.com/) to 'regular' bsd sockets, which is doing good, and i'm planning to use parts of the lua engine from Lua-RTOS as a 'command shell', if I may....

I'll post a comment here, if there's anything to look at.

[allogic]

After trying this myself, I came to the same conclusions as @mgliewe.

I got it working in the end (idf v4.4.3), but some parts of the filesystem aren't working as they should.

• chdir and getcwd inside idf/components/newlib/realpath.c in particular, their implementation is a bit misleading.
• mounting a filesystem to a specific mount point doesn't mount it at the specified path eg. mount "/spiffs" -> "/". It seems to be mounted but only on the original filesystem path. Like ls /spiffs shows the files correctly bot not ls /. This holds true for other filesystems I've tried. (lfs, fat)

Other components certainly don't work as they should either, but I haven't tested those yet.
Since I've spent far too much time on testing this, I would really appreciate some response on this topic.

[the0ne]

@allogic please upload what you have so far to a fork so we can see what's the problem.
Thanks in advance!

[allogic]

Here is a fork with my described problems https://github.com/allogic/Lua-RTOS-ESP32

[the0ne]

Wow, thanks a lot @allogic 😎

@mgliewe @a2800276 @jolivepetrus @TotallyNotSethP @nlw0 @ar055 @wasn-eu @KernelErr looks like Lua-RTOS has been ported to idf v4.4.3. Who'll step up and find out what it needs to be finalized? Your turn now 😊