Libervia progress note 2021-W18

goffi 3 years ago jabber-xmpp-en SàT Libervia project libre Libervia progress SàT progress

Hi,

again, lot of things have happened since last progress note, so I'll only talk about major changes here.

"Salut à Toi" is now "Libervia"

The project has been renamed to "Libervia". Even if I personally loved the former name (which was a reference to a French punk band song, an which could be translated to "hi to you", a nice fit for a communication tool), it proved to be hard to pronounce and remember for non French speakers, and the many names of frontends and project components were confusing. The name change has been discussed for long in the association, but the new ActivityPub/Pubsub end-to-end encryption project accelerated things: after a talk with NLnet, we decided to move forward on this so project name would not change in the middle of its development.

After doing a quick poll, we confirmed that "Libervia" (which was formerly the name of the web frontend only) would be the new name.

All parts are now named in straightforward way: "Libervia Backend", "Libervia Web", "Libervia Desktop/Mobile" (currently the same Kivy frontend for both), "Libervia TUI" and "Libervia CLI", with matching executable names (libervia-backend, libervia-web, libervia-desktop, libervia-tui, libervia-cli also aliased as li). The former names are kept internally and as aliases.

The non-profit (French "loi 1901") association behind it stays with the name "Salut à Toi".

This renaming has involved a lot for work, it took weeks to update code, web sites, doc, etc. and according to our statuses, we had to make a general assembly to discuss this decision. It's still not fully finished (notably the official web site URL is still https://salut-a-toi.org, while https://www.libervia.org is currently used for the demo server), and source code repositories have not been modified for the moment, but most of the renaming is done, and you can now reference the whole project as "Libervia"

Official Website Update

Following the changes in Libervia Web themes, the official website one has also been updated and is now based on Libervia's Web Bulma theme. The news now links to my personal blog as it is where you'll have most up-to-date informations about Libervia development (and the former page was broken). Tickets/Bug tracker is now directly accessible from the official site, as it makes more sense to have it there. It's still accessible from goffi.org, and thanks to its decentralised nature, it's usable transparently on both locations.

I've also temporarily disabled account registration on the bug-tracker due a wave of spammy accounts. I will have to put in place a protection for that, but I'm reluctant to use popular non-libre options.

Flatpak and Docker

While working on the renaming and website, I've updated the Flatpaks (they were really outdated), and Docker images. Flatpaks is for now using a specific dev repos, but I hope to see Libervia on Flathub after the release.

I've created Docker images and Docker Compose file to run quickly a local demo of Libervia Web, you can see the instruction on the Official Website.

Ideally, I would like to also create Snaps, Appimages, Nix packages, etc. But I'm lacking time (Flatpak and Docker are already too much time consuming) and prefer to focus on the code rather than on the packaging, help is more than welcome though.

User Friendly URLs

As you may have noticed on the last blog posts, URLs are now more user friendly:

A blog post is referenced using its item ID, and previously a unique ID was used for that, which is relatively long and doesn't give any information about the content, but is necessary to avoid conflict (writing a blog post with an existing ID will overwrite the previous one).

To make it more pleasant, a URL friendly extension was then added, and not used to retrieve the item, so in the example above, www.goffi.org/b/LFMqr7xC2aNf4MDgkbamBY links to the same blog post as www.goffi.org/b/LFMqr7xC2aNf4MDgkbamBY/sat-progress-note-2020-w53. The resulting URL is long and not easy to read, but the item is unique.

The new behaviour directly use URL friendly item IDs, and to avoid conflict, a short random suffix is appended (on the example above, QGqK is the suffix). After some tests, the collision risk for a short suffix like that is not that high (I've tested millions of IDs without collision), and it may anyway happen only if 2 blog posts have the exact same title, so the risk is very low. The resulting URL is more pleasant.

This URL friendly ID is used by default when a blog post is created, but it can be deactivated if user_friendly_id is set to false in blog post metadata, or by specifying manually an item id.

To accompany this change, a new Libervia CLI rename subcommand has been added to li blog and li pubsub, which will change the ID of an item. As there is no standard rename operation in XMPP Pubsub, this is done by copying the item to the new ID, then delete the former one in case of success.

Navigation Helpers in Libervia Web

It was not really easy so far to know where we were in Libervia Web. To help with this, the selected menu is now shown activated, and a breadcrumb has been added.

The breadcrumb is only shown when there are at least 2 elements to show (i.e. not on root pages). It is generated automatically by default, but can be customised with specific label, sub-elements, or even icons, like in the file sharing screenshot below:

Libervia Web 0.8 Breadcrumbs Screenshot

Blog Editor

As it was not possible anymore to write a new blog item with Libervia Web, I've made a blog item editor, which is relatively basic for now, but working. If JavaScript is activated, you'll get a tags editor, preview, and autosaving:

Libervia Web 0.8 Blog Editor Screenshot

File Sharing Quotas

One last missing piece I was needing before release was to put in place quotas on the file sharing component, this is now done.

Indeed, this component doesn't work with a per-file limit like most others do, but with a per-user quota, and you can upload any file size you want at long as you're not over quota.

Release to come

It's more than time to think about the release. I wanted to improve the chat notably in Libervia Web where it's still really basic since we moved out from the former frontend, but finally I've decided to report this to next release, as I plan to refactor messages handling, and for now I need to concentrate on the ActivityPub gateway.

So I'll soon prepare a beta version, and plan to do the release in a couple of weeks. I'll do bugfix on the 0.8 version during this time, but will avoid any important new development.

ActivityPub gateway project

With all the work done above (and other things, I've not mentioned everything), I've been late to start working on the ActivityPub project, but now I can focus on it. The first task is about developing a Pubsub cache as Libervia is currently getting its data for Pubsub related feature directly from the services.

Beside the obvious speed improvement, having a local cache will give the possibility to do data search/manipulation (such as doing Full-Text Search when the Pubsub service doesn't implement it, or doing feature-specific data analysis), handle message received unordered, allow to keep decrypted data when received from e2ee items, etc.

So far, SQLite was used for data storage in Libervia, by using Twisted's adbapi and custom semi-automatic schema update/data migration. It has been working relatively well so far, but it's no pleasant to maintain.

Fortunately, SQLAlchemy has recently added support for AsyncIO, thus it can now be used in Libervia. This is great, as SQLAlchemy is popular and rock solid, so I've decided to go with it. This will open the possibility to use other backends (like PostgreSQL), and refactor Libervia to use SQLAlchemy's ORM. Logically, Alembic will be used for data migration, which should make database modifications easier.

Such a cache will make possible to implements things like items discovery based on categories (or search by "hashtags" as it named in other software).

That's all for this note, see you soon.

E

errormovim 3 years ago

E

edhelas 3 years ago

M

matlag 3 years ago

debacle 3 years ago