KDE PIM/Akonadi - KDE TechBase

KDE PIM/Akonadi - KDE TechBase
Jump to content
From KDE TechBase
KDE PIM
Status
Supported Payload Types
Akonadi Braindump
Ideas/notes etc. on various open issues in Akonadi.
Akonadi Standard Actions
Idea: Have something like KStandardAction for Akonadi that not only includes the representation of the action but also its state management and the actual operations. Like libakonadi that should be splitted into a generic, type-independent part and be extensible for type-specific actions. This will enable code sharing among many applications and guarantee consistent actions everywhere.
State management: watch selection models of a collection and/or item model.
Use KXMLGUI for context menus in standard views to allow easy extensibility with custom actions.
Generic actions:
new collection [done]
new virtual collection
delete collection [done for single selection]
copy collection [done]
cut collection
paste collection [done]
synchronize collection [done for single selection]
synchronize resource
synchronize everything
show collection properties dialog [done]
delete item(s) [done]
copy item(s) [done]
cut item(s)
paste item(s) [done]
paste native data
tag item(s)
comment item
rate item(s)
configure resource
add resource? (would need a type parameter, etc.)
delete resource
manage local subscriptions [done]
move to submenu
copy to submenu
The list is definitely long enough to make this worthwhile.
Resource API issues
Notes/ideas/complaints about the Akonadi::ResourceBase API:
Extremely error prone:
Scheduler dead-locks when a requested operation is not correctly announced as finished, esp. a problem in error cases.
Using the result methods multiple times or when not requested asserts
Item streaming is missing, requiring all data to be in memory at once
Non-incremental updates need to know the results of ItemSync to not have to provide all data, even for already existing/unchanged items.
API / BC issues
Smaller stuff that should be fixed before the ABI freeze and is not yet listed above:
Cleanup the D-Bus format used by NotificationMessage
No mentioning of "IMAP" in the public API, we are not using IMAP
Naming and installed location of libraries, headers and executables (see discussion on kde-pim ML)
Payload part labels are QStrings, attribute types are QByteArray
Deployment issues
Multiple access: Should multiple Akonadi instances' mysqlds access a single set of data files the mysql will likely corrupt the data. This can happen in any NFS+YP installation where users can log onto any machine and access shared homes. MySQL relies on filesystem locking to prevent multiple access.
MySQL multiple instance docu
InnoDB tables should not be used on NFS
NFS speed: MySQL documentation recommends against locating its data files on network shares.
AppArmor: Distros' AppArmor profiles prevent MySQL from writing outside its defined data directory (usually /var/lib/mysql). This is a problem at least with *buntu and openSUSE. These will need to be adapted. It is possible to daisy-chain profiles so that MySQL started by Akonadi receives a different profile to MySQL running standalone. An empty profile has all rights. Another possibility is to adapt the general MySQL profile so it can write to ${XDG_DATA_HOME}/akonadi(usually ${HOME/}.local/share/akonadi). Both support for profile chaining and ${HOME} may depend on the version of AppArmor installed. What about SELinux?
Backup: MySQL data files should not be backed up without telling the mysqld process, otherwise a corrupt backup will be made. LOCK TABLES and FLUSH TABLES at the least. A dump can be made or mysqlhotbackup may be used in some circumstances. We should consider sysadmins' backup techniques when planning/promoting Akonadi, as a simple rsync cronjob with running Akonadi will not work.
MySQL backup advice
Documentation
Avoid examples using KJob::exec() due to its problematic side-effects.
Examples in the ItemCreateJob docs (and maybe others) operate on items in the root collection which does not work at all
Resurrect the server API docs (now at
).
Range Queries, Sorting
Range queries require indexes on certain parts of the payload so it becomes i.e. possible to request only the events that are currently visible in the view. The same is required for server-side sorting so we can request the 200 latest emails sorted by date.
The feature could be implemented by creating temporary virtual collections containing the search result.
A calendar would i.e. "search" for all events of the current month, which would already solve things such as
not getting notifications for items we are not interested in.
This search folder would however have to be modified with a new query as the user scrolls in the view.
Workspace integration
Email notifier, based on UI of LionMail [
[1]
], in more detail:
Write (or lend from Kontact Touch) ProxyModels to display email collections: new / unread and important, combination thereof
Plasma Quick widgets for individual emails and collections
UI integration
Konfiguration
Drag and Drop (from Collection onto desktop)
Theming / Animations
Streamlined notifications (MUST NOT get in the way)
Activity support (notify off for certain activities, only selected folders for a given activity, etc)
The current Lion Mail code uses a dataengine to retrieve data from AKonadi. This needs to be ported to using one of the AKonadi models which are also used in Kontact Touch.
Agent monitoring and control
Enable/disable logging?
Plasma applet for notes / sticky notes
Resource progress, e.g. like KIO progress
Drag&Drop of Akonadi items: drop target should be able to retrieve payload
Akonadi FAQ
This FAQ primarily deals with technical and design questions, if you have questions about using Akonadi (such as "I get error X when starting Akonadi"), please refer to
userbase.kde.org/Akonadi
Where do I find the Akonadi config files?
~/.config/akonadi/
Where does Akonadi store my data?
Akonadi merely acts as a cache for your data, the actual content stays where it has always been, .ics/.vcf/MBOX files, local maildirs, IMAP- and groupware servers. There is only a limited amount of data stored exclusively in Akonadi:
Data not supported by the corresponding backends, such as email flags in case of maildir/mbox. This is comparable to KMail's binary index files stored alongside these files in pre-Akonadi times.
Internal meta-data used by application or resources, such as information about the last synchronization with a backend or translated folder names.
Data that has been changed while the corresponding backend has been offline and has not yet been uploaded.
Where can I find the Akonadi database?
~/.local/share/akonadi/
Where can I find the source code?
Code for the respective parts of Akonadi is in multiple code repositories:
The Akonadi server can be found at the
src/server subdir of the akonadi main code repo
The Akonadi client libraries are distributed over the
other subdirs of /src in the akonadi main code repo
Various agents, resources, and much other misc stuff is in
the kdepim-runtime code repo
the Akonadi console is in
the akonadiconsole code repo
More related repos can be found in
list (TODO: list here)
Where can I find documentation?
General documentation, including a design overview, is on
KDE Community Wiki
The documentation is mainly in the code itself, in the form of doxygen comments. You can find the generated documentation on
api.kde.org
, for example:
Documentation for the KDE Akonadi client library is
here
Documentation about the server is
here
Which DBMS does Akonadi use?
Akonadi currently supports SQLite, MySQL and PostgreSQL. Basically, every database that is supported by QtSQL can be used, requiring minimal changes in the code at most. However, not all of them provide the features needed by Akonadi (see next questions).
Why not use MySQL/Embedded?
We tried that as well, there are two reasons for not using it: No support for the InnoDB engine (which we need for transaction support) and poor availability (only OpenSUSE provided usable packages, needed a patched QSQL driver).
Do I need a running MySQL server?
No. Akonadi starts its own local MySQL server (unless configured otherwise, see next question). All you need is having the 'mysqld' binary installed at runtime (usually found in the mysql-server package of your distribution).
Can Akonadi use a normal MySQL server running on my system?
Yes, it can. You find the corresponding settings in
~/.config/akonadi/akonadiserverrc
Can I connect multiple Akonadi instances to the same database to share my data between different machines?
That does not work unfortunately. Akonadi does not store all relevant data in the database but also uses the file system for configuration and large payload data for example. Also, there is no mechanism to ensure multiple instances have exactly the same version and exactly the same agents and plug-ins installed etc., all of which would be necessary to work on the same database. Finally, there is no notification system to inform the other instances about changes which endangers consistency since the Akonadi server contains internal caches of data in the database. If you want multiple instances to synchronize, use a groupware server (not as bad as it sounds, Kolab for example works with many normal IMAP servers).
If you try this despite the warnings, be aware that there is no safety mechanism in place to prevent you from doing that and you will likely mess up your data in funny ways.
I don't like a database server because of backups/running on NFS/etc.
See section "Deployment Issues" above, we are aware of that and working on it. Some of these, like backup/restore are already implemented. But please be aware that most of this issues also existed before (eg. with KMail's custom binary indexes).
Can a single Akonadi instance be used by multiple users?
No. There has to be one Akonadi server instance per user. However, it is possible to use a shared database server.
Can I access the Akonadi server on a remote machine?
No. Akonadi is not a groupware server. It's a local cache only.
What's the differences between Akonadi and EDS?
EDS (Evolution Data Server) is limited to contacts and calendar data, Akonadi is type-independent. Especially, Akonadi is designed to also handle high-volume data such as email. Akonadi and EDS also differ largely in their access protocol on a technical level (Corba/D-Bus vs. local socket with IMAP-like protocol/D-Bus) and also on a protocol level (type specific vs. generic).
How do I create a collection?
From the developers point of view, there is Akonadi::CollectionCreateJob for that, from the users point of view, most applications allow that, eg. Mailody or akonadiconsole.
However, there is one limitation: Top-level collections can only be created by resources, not by normal applications. The reason for that is that every object (collection or item) is supposed to have an owning resource, that is a resource that is responsible for storing and retrieving that object. There is an ongoing discussion to remove this restriction though.
So, if you want to create a collection eg. for testing purposes, add a resource first. Most Akonadi applications offer an option to do that, a module for KControl is planned as well.
How do I disable automatic migration from KDE's traditional framework?
The migration tool is controlled by standard KDE configuration file called
kres-migratorrc
Distributors or system administrators wanting to disable the automatism will probably want to that globally, e.g. by editing the installed default configuration file, or by using KDE's configuration hierachy and using a profile config between that and the user level.
The quickest way to deactivate it for one user account only is to use KDE's
kwriteconfig
tool to set the respective configration value with a simple copy&paste of the following command:
kwriteconfig --file kres-migratorrc --group Migration --key Enabled --type bool false
Please note that at some point KDE applications such as Kontact, KOrganizer, KMail will be using Akonadi directly, at which point migration either has to be enabled or performed manually.
As of KDE 4.4 (and its development releases and when building from SVN trunk) this already applies to KAddressBook and KMail's address book access.
How do I completely disable Akonadi startup?
Akonadi will be automatically started by any Akonadi-enabled application . If you don't want to start Akonadi at all, you can't use any of these applications (see the list of
Akonadi-enabled applications
). Also note, that some Plasma widgets, such as the "Digital Clock" applet uses Akonadi.
What is the relation between Akonadi and OpenSync?
Akonadi and OpenSync focus on different aspects and complement each other. Akonadi provides a unified way to access PIM data for applications and a framework to implement powerful connectors to varies data sources. OpenSync focus is on syncing two sets of PIM data.
An Akonadi plugin for OpenSync is currently under development, allowing to sync PIM data available through Akonadi with any other system supported by OpenSync, especially mobile phones.
When should I use Akonadi?
More precisely, when should you use for your application specific data instead of eg. just using a local file directly.
Akonadi is especially useful when you need one the following:
Different backends for your data, like eg. a local file and a remote server. Akonadi provides a unified interface for application developers to access your data independent of the actual backend.
Caching and change replay of remote data. Akonadi has support for that built in, giving you free offline support for any remote backend.
Desktop-wide sharing of your data. As soon as more than one application (say your main applications and a plasmoid) accesses the same data you need to deal with locking, conflict detection, change notifications, etc. - or let Akonadi do that for you.
However, if you are just looking for a simple way to store your application data without needing one of the above, using Akonadi usually means more implementation work for relatively little gain.
Akonadi needs too much space in my home directory!
An empty, unused Akonadi database needs about 100 Mb of disk space. This is due to the MySQL InnoDB log files which work similar to a journal in a modern file system. These files are constant in size and independent of the actual data stored in Akonadi. The default size is optimized for performance on average desktop hardware where the use of 100 Mb of disk space is no problem. In other cases, such as multi-user systems or embedded devices, this default might not be optimal though.
The default size can be configured, globally or per-user. The global configuration file can be found in $PREFIX/share/config/akonadi/mysql-global.conf, the per-user file is in ~/.config/akonadi/mysql-local.conf and overwrites settings of the global file. In any of these files, you can change the settings
innodb_log_file_size
and assign it a smaller value than the default (64M).
For this setting to take effect you need to restart Akonadi. With older Akonadi versions (<=1.1.1) you might need to manually remove the InnoDB log files from ~/.local/share/akonadi/db_data for this change to take effect. The log files do not contain data after a clean shutdown and thus can safely be removed while Akonadi is not running.
An alternative approach especially for multi-user systems might be the use of a single, global MySQL server instance.
My Akonadi resource seems to randomly hang/stop working!
A very common problem of resources based on Akonadi::ResourceBase are "unguarded exit paths" from one of the methods you have reimplemented there. See the following example:
void
MyResource::retrieveItems
const
Collection
collection
if
someError
return
itemsRetrieved
myItems
);
In case of an error you leave retrieveItems() without telling Akonadi::ResourceBase that you are done with the task. Therefore, it is assumed the requested item retrieval takes a bit longer (which is not uncommon for resources for remote backends, results typically come in in a result slot connected to a job class for example) and waits until you announce the task is finished.
The following example does it correctly:
void
MyResource::itemAdded
const
Akonadi
::
Item
item
const
Akonadi
::
Collection
parent
if
noNetwork
// transient error
deferTask
i18n
"Offline, will retry later."
);
else
if
item_not_valid
// permanent error
cancelTaks
i18n
"Got invalid crap, can't store that."
);
else
// store the item here
Item
newItem
item
);
newItem
setRemoteId
my_new_remote_id
);
changeCommitted
newItem
);
The following methods require explicit notification that a task has been completed:
retrieveX
any method indicating changes, ie. itemAdded|Changed|Moved|Removed(), collectionAdded|Changed|Moved|Removed()
any custom task scheduled with ResourceBase::scheduleCustomTask()
Refer to the API documentation of Akonadi::ResourceBase for the various ways to do that correctly, there are a few different ways, depending on the current task:
Successful completion with convenience features, eg. changeCommitted(), itemsRetrieved() etc.
Error cases with convenience features, eg. cancelTask(), deferTask()
Only indicate completion, with everything else done manually, eg. changeProcessed(), taskDone()
To confirm a resource is affected by this problem, the "Resource Schedulers" tab of akonadiconsole is very useful (needs to be enabled in the context menu first, causes too much slowdown otherwise). It shows the state of the internal task scheduler of your resource, allowing you to spot stuck tasks.
Information for Developers using Akonadi
References to information for developers using or extending Akonadi.
Tutorials
Application Development
Resource Development
Serializer Plugin Development
Creating accountwizard packages
Application Porting
Documentation
Akonadi Testing Framework
Akonadi Development Tools
Akonadi Firstrun
Akonadi Trashhandling
Debugging Akonadi IMAP Resource
Client Library API documentation
Akonadi Multi-Instance Setup
Projects/PIM/Akonadi/Architecture
Contact & Getting Involved
#akonadi
IRC channel on Libera Chat
[email protected]
mailing-list
Akonadi Internals
References to information for developers of Akonadi itself (the above section is of course also relevant for you).
Documentation
Akonadi Release Howto
Akonadi Database Internals
Akonadi Server API documentation
Meeting Notes
Osnabrueck 7
Osnabrueck 8
Akonadi/KDE PIM pre45 sprint 2010
PIM Sprint 2014
Retrieved from "
Category
PIM