Posted in 2011
SyncEvolution 1.2.1 released
- 28 November 2011
Maintenance release with various bug fixes. * GTK UI + config: fix “custom server” setup ([BMC #13511](http://https://bugs.meego.com/show_bug.cgi?id=13511)) When the “default” config template (= ScheduleWorld) was downgraded to “not consumer ready” in SyncEvolution 1.1.0.99.1, setting up a custom SyncML service in the GTK UI stopped working because the UI wouldn’t show the “not consumer ready” config. The problem described above is deterministic and fixed now. Initially the problem seemed to be random. So perhaps there is also another, related issue. * phone sync: delete<->delete conflict + phone calendar+todo sync ([BMC #23744](http://https://bugs.meego.com/show_bug.cgi?id=23744)) When deleting an item on phone and locally, the next sync failed with ERROR messages about “object not found”. Retrying the sync then worked. * Nokia: prevent accidental usage of “calendar” or “todo” sources Nokia phones use a combined “calendar+todo” source for syncing. The “calendar” and “todo” sources also exist because that is where local databases are configured. In such a setup, syncing always has to use “calendar+todo”. For example, to refresh from the Linux desktop to the phone, use: –sync refresh-from-server
deb http://downloads.syncevolution.org/apt unstable main These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/evolution). In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
State of the union, version 1.2
- 25 October 2011
With SyncEvolution 1.2 released and work on 1.3 under way it is a good time to take a small break and reflect on the state of the SyncEvolution project. Features ====== This is meant to give an idea of what can be done with SyncEvolution already. For a more complete list of features, see the documentation about [backends](/documentation/backends), [frontends](/documentation/frontends) and [engine](/documentation/syncevolution-and-synthesis-syncml-engine). This section introduces the key ideas, some of the supported protocols + backends and how they work in combination with specific peers. Data Synchronization ——————————- The main purpose of SyncEvolution is the synchronization of Personal Information Management (PIM) data (contacts, events, tasks, notes). This intentionally does not include email and messages in general because those are sufficiently different to require different solutions. SyncEvolution provides *real* synchronization of PIM data: * Items (= contact, event, …) may have different properties and representations on both sides of a sync (heterogeneous environment). * Items can be modified at any time, even while offline (more than just online access to a server). * Also works between devices, without involving a server. * A database (= a set of items) can be synchronized to one or more peers (other devices, servers). For example, the same address book on a desktop can be synchronized with a mobile phone and a laptop. * SyncEvolution supports arbitrary number of databases. However, many peers (in particular SyncML servers) are more limited and only provide access to one addressbook, calendar, etc. For each peer, SyncEvolution needs to remember the previous state so that the next sync can be limited to only those items which have changed. The underlying assumption is that there is no cycle in the connections between multiple sync peers. For example, “phone <-> laptop, laptop <-> server, phone <-> server” is bad because a new item created on the phone will go to the laptop, from there to the server and then come back as new item to the phone, resulting in a duplicate which then repeats the process ad infinitum. Each SyncEvolution instance only knows the peers it is set up to talk with and thus cannot detect the problem. To break this cycle reliably, each item must have a unique ID that is assigned to it once when the item gets created. For ad-hoc synchronization to work without constraints on the topology, this ID must be supported by all involved peers. Unfortunately, calendar and contact data either doesn’t have such an ID or it is not supported. Without such an ID, each request to add a new item must be checked against all existing items based on some key properties to find out whether it is a duplicate. This is slow and error-prone (“Is ‘Doe, John” the same person as ‘John Doe’, or is one the father and the other a child or namesake?”). *TODO* “[ad-hoc synchronization](https://bugs.meego.com/show_bug.cgi?id=23760) between SyncEvolution instances”: * Teach Evolution Data Server to create and preserve a unique ID for contacts (the vCard 3.0 UID gets overwritten at the moment). * Improve the SyncEvolution<->SyncEvolution sync such that * it uses the unique ID to speed up sync between peers which sync against each other for the first time and * checks the ID to find duplicates in following syncs. *TODO* “[ad-hoc synchronization](https://bugs.meego.com/show_bug.cgi?id=23760) between arbitrary peers”: * Do the duplicate detection based on item content for each add request. Testing ———- SyncEvolution has an extensive test suite which is run regularly (used to be nightly, currently triggered manually). It contains unit tests for various aspects of SyncEvolution and does real interoperability testing with different SyncML, CalDAV and CardDAV servers. See [the SyncEvolution 1.2 test report](/blogs/pohly/2011/syncevolution-12-released#comment-525) for an example. More CalDAV/CardDAV servers were added in the 1.3 branch. Keeping this testing going and analyzing/reporting problems is an on-going activity. It is needed to avoid regressions and achieve higher quality in the releases, which are built as part of these test runs. There are additional ideas. *TODO* “improve nightly testing” * Only the sync code is currently checked under valgrind. Also run [syncevo-dbus-server under valgrind](https://bugs.meego.com/show_bug.cgi?id=5609). * Run [tests in MeeGo (or soon Tizen) chroot](https://bugs.meego.com/show_bug.cgi?id=8684). * Automatically [integrate and test](https://bugs.meego.com/show_bug.cgi?id=735) branches which are considered ready for integration. * Test [syncevo-http-server](https://bugs.meego.com/show_bug.cgi?id=1367). Could do [suspend/resume tests](https://bugs.meego.com/show_bug.cgi?id=1009) against that. SyncML + Engine ————————— The Synthesis engine which provides SyncML is one of the best in the industry. It has extensive support for data modeling/conversion and supports suspending a session and resuming it later. The same engine is also used to synchronize between two backends internally. This is how synchronization was added for CalDAV/CardDAV, protocols which themselves only provide online access. This works reasonably well, but there are also quite a few limitations. *TODO* “[libvxx = refactoring of the Synthesis code base](https://bugs.meego.com/show_bug.cgi?id=23761)” * make data conversion available outside of a sync session * decouple sync engine from SyncML and SyncML message encoding/decoding * support items which are a set of items: important for CalDAV + ActiveSync, because those combine all VEVENTs with the same UID in one item; certain transformations cannot be done in the engine at the moment because they depend on access to all related items at once and that is not how they are handled at the moment * more flexible session handling: instead sending changes in one direction, then back and then stopping, allow the session to continue until both sides are in sync; required for CalDAV where storing an event might lead to further changes that have to be sent back *TODO* “[push sync](https://bugs.meego.com/show_bug.cgi?id=23762)”: * react to local or remote changes immediately (instead of polling at long intervals) and/or * make sync sessions without changes more efficient (in particular when polling) *TODO* “[better credential handling](https://bugs.meego.com/show_bug.cgi?id=23763)” * When creating multiple configurations which need the same credentials (Google CalDAV and SyncML, for example), the username/password needs to be set separately. Replace with a mechanism where both configs only contain a pointer to shared credentials. * Also support an external system component which does the authentication without ever returning username/password to SyncEvolution. Depends on having such a component. This could be used to use a CalDAV/CardDAV server as backend for a SyncML server. *TODO* “[use CalDAV/CardDAV inside SyncML server](https://bugs.meego.com/show_bug.cgi?id=23764)”: * In the backend, check databaseuser/password before falling back to the context’s username/password. * Support shared credentials, to avoid having to configure them in each source. At the moment there are situations where SyncEvolution cannot determine what the right resolution for a failed synchronization is. It has a built-in backup mechanism and can ask the user for assistance, but ultimately it would be better to not bother the user. This can be achieved in some cases by simplifying the problem: * The peer (most likely a service on the Internet) must be able to store all data. * There is a separate local database for each database on the peer. In that case it is acceptable to wipe out the local data and restart with the data stored on the peer. *TODO* “[automatic error recovery](https://bugs.meego.com/show_bug.cgi?id=23765)” * Implement the necessary policy in SyncEvolution (“peer wins”). * Check whether this interferes with libfolks (local IDs will change, data added by libfolks to a contact might get lost). CalDAV/CardDAV ————————– Works reasonable well and passes automated testing against a variety of servers (Apple Calendar Server, DAViCal, Google Calendar, Yahoo). But there are some know limitations, like meeting invitations being sent by Evolution and the CalDAV server. *TODO* “[better support for meeting invitations](https://bugs.meego.com/show_bug.cgi?id=23766)” * Find a way to suppress sending of meeting invitations on the CalDAV server when Evolution already sent one, *or* * suppress sending of meeting invitations in Evolution for calendars which are mirrored in a CalDAV server. * When storing a meeting and/or the server does not return an ETag, retrieve the possibly modified item from the server and store the modified item locally. Right now the more possibly automatically modified data on the server gets ignored. Depends on the engine improvements mentioned above. *TODO* “[handle concurrent changes](https://bugs.meego.com/show_bug.cgi?id=23767)” * Use ETags to avoid modifying more recent data on the server. *TODO* “[CalDAV attachments](https://bugs.meego.com/show_bug.cgi?id=23768)” * At the moment, attachments are not supported at all and even may get lost. Need to preserve them and perhaps even support the more efficient “managed attachments” that are currently being discussed by the CalConnect consortium. *TODO* “[WebDAV: use sync extension](https://bugs.meego.com/show_bug.cgi?id=23769)” * Each sync session must list the entire collection (= retrieve path names and ETag) to determine new and modified items. This can be done more efficiently by using the sync extension defined and implemented in the Apple Calendar Server. *TODO* “[CalDAV tasks = VTODO](https://bugs.meego.com/show_bug.cgi?id=24893) * The CalDAV backend is limited to exchanging VEVENTs. It might also not handle collections well (or at all) that contain both VEVENTs and VTODOs. ActiveSync —————- This was not included in SyncEvolution 1.2 although it was already quite usable. Work on it is still going on. The main issue with ActiveSync is the limited data model specified as part of the protocol. Contacts are only allowed to have a fixed number of certain phone numbers and addresses, which is a limitation that neither Evolution nor Google Contacts have. There is no good way to handle these limitations except educating the user about them, or enforcing the same constraints locally by modifying the app which creates and modifies contacts. The ActiveSync calendar format does not support detached recurrences properly (“you are invited to a specific instance of a meeting series”), although Exchange internally does. *TODO* “finish initial ActiveSync support” * [detached recurrences without parent](https://bugs.meego.com/show_bug.cgi?id=22831): * investigate receiving multiple detached recurrences in multiple items with the same UID in each (the Exchange workaround for the ActiveSync calendar format limitation): might break SyncEvolution * implement the “stand-alone detached recurrence” support (either do it like Exchange does or better, create a fake parent event) * test [concurrent item changes](https://bugs.meego.com/show_bug.cgi?id=23770) while a sync runs (supposed to work, but without a test case it is hard to be sure) **[updated]** *TODO* “ActiveSync performance improvement” * Writing changes to the server is done one change at a time. Could be improved considerably by batching changes. Depends on [core engine improvements](https://bugs.meego.com/show_bug.cgi?id=23967). *TODO* “[ActiveSync push sync](https://bugs.meego.com/show_bug.cgi?id=23771)” * enhance activesyncd and SyncEvolution (see engine above) to react to server-side changes with minimal delay *TODO* “[ActiveSync calendar attachments](https://bugs.meego.com/show_bug.cgi?id=23772)” * At the moment, attachments are not supported at all and even may get lost. Need to preserve them. Not sure how to do it efficiently. Google Calendar ———————— Overall Google Calendar syncing works well with CalDAV (I’m using it myself) as long as one does all meeting scheduling in the Google web interface. But there are some known issues, most of them on the server side: * [stand-alone detached recurrences cannot be accessed, SyncEvolution workaround only works for some cases](http://code.google.com/p/google-caldav-issues/issues/detail?id=58) * [meetings removed by SyncEvolution](http://code.google.com/p/google-caldav-issues/issues/detail?id=59) - not sure yet whether this is better fixed on the server or client side * [cannot updated/delete meetings although the organizer](http://code.google.com/p/google-caldav-issues/issues/detail?id=60): SyncEvolution is allowed to create an event, but then cannot update or remove it * [cannot remove detached recurrence](http://code.google.com/p/google-caldav-issues/issues/detail?id=61): breaks the testLinkedItemsRemoveNormal tests Google Contacts ————————- Works via SyncML. Google’s support for SyncML is very incomplete (many properties not supported, for example birthday). *TODO* “better Google Contacts support” * [Test ActiveSync](https://bugs.meego.com/show_bug.cgi?id=23773) and/or * write [backend based on Google Data Protocol](https://bugs.meego.com/show_bug.cgi?id=23774) - might be the best solution, because ActiveSync also has limitations (see above). Apple Calendar Server, DAViCal ———————————————- Done via CalDAV/CardDAV. No know issues with these peers. Yahoo Calendar + Contacts —————————————– Works well, when it works; unfortunately the number of requests per 24 hour period is so limited that the tests cannot complete without running into a 503 “Service Unavailable” error. *TODO* “[Yahoo token authentication](https://bugs.meego.com/show_bug.cgi?id=23775)” * Currently SyncEvolutions uses normal HTTP authentication. Yahoo also supports another, token based authentication mechanism for approved apps. Get SyncEvolution approved and implement that other authentication mechanism. Hopefully that’ll avoid the 503 error. Phones ———– SyncEvolution can synchronize against phones if those phones support SyncML via Bluetooth. This is a common feature among older feature phones but most (all?) Android phones and iPhones don’t support it. Testing of this feature is limited and there is no list of phones which are guaranteed to work. **[updated]** Synchronization with Android/iOS is possible by installing a third-party SyncML client (like the ones from Synthesis) and configuring the phones to use a [SyncEvolution HTTP SyncML server](/wiki/synchronizing-evolution-http-howto). *TODO* “[contact sync via PBAP](https://bugs.meego.com/show_bug.cgi?id=23776)” * Write a backend using PBAP, the only (?) protocol supported by Android and iOS for address book access. Not very good for real syncing, but at least one-way sync should be possible. GTK UI ———– The GTK “sync UI” is the main user interface for SyncEvolution in MeeGo and Linux in general. It is included in the source and binary distribution archives. It supports configuring syncing against SyncML services (extensible via configuration templates) and phones which support SyncML. Configuring the latter is integrated into the GNOME Bluetooth applet, which invokes the sync UI. The UI also has “emergency recovery” support which allows the user to restore from the automatic backup and/or choose between different recovery operations after a failed sync: * continue with local data * continue with remote data * try to merge both (“slow sync”) *TODO* “[port to GTK3](https://bugs.meego.com/show_bug.cgi?id=23094)” * The sync UI is currently using GTK2. It needs to be ported to GTK3. The goal is to put the GTK2 version into maintenance and continue with the GTK3 version. *TODO* “[configure local sync, choose databases](https://bugs.meego.com/show_bug.cgi?id=23777)” * CalDAV/CardDAV and ActiveSync cannot be configured yet with the UI. In this context it becomes important to let the user choose local and remote databases. Would be useful to have, although there is a certain overlap with the Evolution integration. Evolution ————- This happens to be the main backend for storing data locally. Other backends could be supported just as well (there’s nothing technical which favors Evolution) if there were developers motivated enough to implement and test them. KDE/Akonadi has come a long way, but seems less active (see KDE/Akonadi and Community below). *TODO* “Evolution backend improvements” * use new APIs in EDS 3.2/3.4 for more [efficient change detection](https://bugs.meego.com/show_bug.cgi?id=23778) * enable [creating databases](https://bugs.meego.com/show_bug.cgi?id=23779) from inside SyncEvolution again (depends on EDS 3.4 (?)) *TODO* “[integrate SyncEvolution into Evolution](https://bugs.meego.com/show_bug.cgi?id=23780)” * Evolution typically supports offline read access with some of its backends. It does not support write access. SyncEvolution addresses that, but needs to be configured and invoked separately. It would be nice to have seamless and transparent syncing from inside Evolution. ActiveSync is getting integrated like that at the moment -> extend that to CalDAV/CardDAV? File backend —————— There is a generic file backend with a 1:1 mapping between a single item and a file in a directory. The format of the local data can be configured. vCalendar 1.0, iCalendar 2.0, vCard 2.1/3.0 and plain text notes are supported already. *TODO* “[iCalendar 2.0 .ics file](https://bugs.meego.com/show_bug.cgi?id=23781)” * Some people synchronize an .ics file by pointing the Evolution backend to it (no longer works after EDS removed support for the file:// URI) or manipulating the ~/.evolution/calendar data (a hack which only works when being very careful). Write a SyncEvolution backend which reads/writes an iCalendar 2.0 file using libical. KDE/Akonadi ——————– The Akonadi backend is available in the source code. Support for KWallet instead of GNOME keyring is also there. A KDE GUI is in development. *TODO* “[test and release binaries](https://bugs.meego.com/show_bug.cgi?id=23782)” * nightly builds needs to be reconfigured to enable building the KDE support * nightly testing needs to include KDE backends, both for unit testing the backends and real combinations with peers; it depends on the KDE developers to do something if issues are found in those tests (if any are found) Manipulating PIM data ——————————– SyncEvolution also is a very capable tool for manipulating databases via the command line. It can list, import/export and delete items. Converting between formats (for example, mirror Evolution contacts as files in vCard 2.1 format) is possible by setting up synchronization with a file backend as peer. *TODO* “improve item manipulation” * the command line syntax is a bit awkward and depends on dummy configurations: [make config+source names optional](https://bugs.meego.com/show_bug.cgi?id=23783) * [convert between formats](https://bugs.meego.com/show_bug.cgi?id=23784) (depends on libvxx engine improvements) Community ======== Lots of users. The feedback from users is often very helpful for improving the software. Not so many developers, though, except those paid to work on SyncEvolution. Ove Kaaven (port+UI for Maemo/N900 and MeeGo Harmattan/N9/N950) and Frederik Elwert (Genesis UI) are the notable exceptions. Sascha Peilicke/Dinesh/Rohan Garg have done some work for KDE/Akonadi, but are not very active in the SyncEvolution project itself. I can only speculate about the reasons for the lack of external contributors: * The number of people who care about PIM sync and storage is small to start with. It’s not one of those sexy areas that gets a lot of attention, although it is arguably very important. * OpenSync still seems to be considered “the” open source sync solution, despite not having a stable release available or even in sight anytime soon. * Perhaps developers also get the impression that problems will be solved anyway, without having to get involved. That is true for some aspects, but definitely not for all. * SyncEvolution development has a certain learning curve (although Ove and Franz Knipp managed to write their backends with very little assistance) and often happens at a rapid pace, which makes it hard for people to contribute small improvements. * SyncEvolution was focused on Evolution and SyncML initially, which might make it seem too limited in scope for some use cases. But the scope has already increased and will increase further, so now would be a good time to check it out again. Many of the TODOs above will not get worked on unless some external developer picks up the mantle and contributes patches. I’d be more than happy to help someone get started. If there is interest, I can also tag bugs in Bugzilla as “easy fixes”. Right now I am not doing that because it would create additional work which would not be justified if there is no-one interested - I know, chicken and egg…
SyncEvolution 1.2 released
- 17 October 2011
The major new feature of the 1.2 release is support for non-SyncML protocols in general and CalDAV/CardDAV in particular. ActiveSync support is in development and will be in 1.3. These protocols are implemented as backends which are combined with other backends by SyncEvolution in a so called “local sync”. The GTK sync-ui does not yet support configuring non-SyncML protocols. See the [README.rst and man page](/documentation/syncevolution-usage) for more information on how to use the new feature via the command line. Properties not supported by SyncML servers can now be preserved locally in two-way synchronization ([BMC #15030](https://bugs.meego.com/show_bug.cgi?id=15030)). This depends on information about what properties a SyncML server supports (“CtCap”), which is typically not provided by servers. SyncEvolution contains a copy of that information for Google Contacts ([BMC #15029](https://bugs.meego.com/show_bug.cgi?id=15029)). Akonadi backend and KWallet support were merged. They are not included yet in syncevolution.org binaries. To use them compile from source. The configuration format was updated to solve a conceptual problem inherited with the legacy property names: the “type” property had multiple, sometimes conflicting roles. For example, setting the preferred data format for sync with one peer might have changed the backend selection for some other peer ([BMC #1023](https://bugs.meego.com/show_bug.cgi?id=1023)). Now “backend/databaseFormat/syncFormat/forceSyncFormat” replace “type”. “type” is still accepted by the command line as alias. Upgrading from releases before 1.2 ———————————– Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Release 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Other changes ————- * Using the –sync-property and –source-property command line options is optional, just specifying the property assignment is enough. * syncevo-http-server was enhanced considerably. See http://syncevolution.org/wiki/http-server-howto * support NetworkManager API >= 0.9 ([BMC #19470](https://bugs.meego.com/show_bug.cgi?id=19470)) * syncevolution.org binaries: now compatible with Debian Testing/libnotify.so.4 ([BMC #22668](https://bugs.meego.com/show_bug.cgi?id=22668)) libnotify is not linked directly into syncevo-dbus-server in the syncevolution.org binaries. Instead libnotify.so.1 till .so.4 (current Debian Testing) are opened opened dynamically and the necessary functions are looked up via dlsym(). Not finding the libraries or the functions silently disables this notification mechanism. * Sync mode is recorded when running in SyncML server mode ([BMC #2786](https://bugs.meego.com/show_bug.cgi?id=2786)). * syncevo-dbus-server automatically stops when some of its libraries are updated and restarts if auto-syncing is on ([BMC #14955](https://bugs.meego.com/show_bug.cgi?id=14955)). * Added code for Buteo, mKCal and QtContacts in MeeGo. Buteo and mKCal were removed again from MeeGo, so the code is obsolete. The QtContacts backend may be still be useful to access items via that API, but for syncing on MeeGo the normal EDS backend is used since MeeGo reverted back to EDS as PIM storage. * “databasePassword” source property: lookup failure in keyring ([BMC #22937](https://bugs.meego.com/show_bug.cgi?id=22937)) The databasePassword also wasn’t looked up at all when doing item operations via the command line. When configuring sources for an HTTP server, the config name typically is just the context (@foo). When using the config in the HTTP server, the config name is the peer inside that context (client@foo). Because the GNOME keyring lookup keys for the “databasePassword” (more specifically, the object name) contained the full config name which was different in both cases, looking up the saved password failed. The solution is to normalize the config name (to accomodate for different ways of spelling it) and use only the context, with @ as before. This will break existing setups where the object name in the keyring (incorrectly) includes the full config name. In that case just configure the source again to set the password anew. * Evolution Calendar: fixed detached recurrence support ([BMC #22940](https://bugs.meego.com/show_bug.cgi?id=22940)) When manipulating a meeting series with more than one detached recurrence certain sequences of operations could incorrectly fail with “UID already exists”. * iCalendar 2.0: must set VALUE in EXDATE (part of [BMC #22940](https://bugs.meego.com/show_bug.cgi?id=22940)) EXDATE has a VALUE parameter, which wasn’t defined in the XML profile. Didn’t seem to matter at all in practice, but wasn’t standard-compliant. * GTK sync-ui: wrap sync service descriptions ([BMC #7199](https://bugs.meego.com/show_bug.cgi?id=7199)) Descriptions of different sync services are not fully visible unless word-wrapping gets enabled. * CalDAV/CardDAV + local storage: avoid empty properties The main motivation for this change is that a recent Apple Calendar server rejects vCards with empty BDAY property. Another reason is that keeping the data as small as possible is desirable by itself. Sending an empty property serves as a hint for the peer that the property is supported. This is not necessary when storing an item in a backend. Therefore this commit disables empty properties for all backends which do not themselves set the m_backendRule Synthesis info value. * Google Contacts: ensure that first/middle/name are set when storing in EDS ([BMC #20864](https://bugs.meego.com/show_bug.cgi?id=20864)) Evolution and the MeeGo UX assume that first/middle/last name are set. That is not the case when a contact is created in the Google Contacts web interface. Such contacts are sent by Google without the N property. SyncEvolution now tries to recreate the name components from the FN string, by splitting at word boundaries and assuming “
deb http://downloads.syncevolution.org/apt stable main These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/evolution). In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
SyncEvolution + non-recursive Automake
- 13 October 2011
Krzesimir Nowak [wrote about his work](http://krnowak.blogspot.com/2011/10/syncevolution-build-system-work.html) on converting SyncEvolution from an autotools project with recursive make to non-recursive make. Definitely worth a read for anyone interested in autotools. The current SyncEvolution master branch (post 1.2) uses that new build system. His conclusion is that the new system is not necessarily easier to understand than the one before (autotools with some preprocessing shell scripts). Partly that’s because SyncEvolution tries to achieve certain things not supported well by autotools (automatically generated version number, avoid listing all files explicitly, backends which can add to the global configure script), partly it is because non-recursive Automake introduces additional constraints (like having to avoid clobbering variables and rules). Helper scripts are still needed, the only difference is how they get called. Either way, what sold me on the idea of a non-recursive make is that on a machine with many cores, like the SyncEvolution nightly test server, compilation is considerably faster because parallel make can spawn more jobs in parallel. Recursive make often had to wait for the completion of compilation in a sub-directory. If memory serves me right, it was more than twice as fast.
SyncEvolution 1.1.99.7 released
- 16 September 2011
Mostly bug fixes again. Some are a bit more intrusive, thus another pre-release. Changes ======= * syncevolution.org binaries: now compatible with Debian Testing/libnotify.so.4 ([BMC #22668](https://bugs.meego.com/show_bug.cgi?id=22668)) libnotify is not linked directly into syncevo-dbus-server in the syncevolution.org binaries. Instead libnotify.so.1 till .so.4 (current Debian Testing) are opened opened dynamically and the necessary functions are looked up via dlsym(). Not finding the libraries or the functions silently disables this notification mechanism. * calendar sync: better handling for add<->add conflicts (partly fixes [BMC #22783](https://bugs.meego.com/show_bug.cgi?id=22783)) When both sides of a sync have added the same event, the sync must determine which one is more recent instead of blindly overwriting always the same side. Such conflicts are typically rare except for enterprise scenarios where meeting invitiations are processed automatically by a groupware (Exchange, Google Calendar/Mail, …) and then the attendee status is updated on one side. SyncEvolution now does the necessary age comparison and preserves the more recent data for most properties. In some properties the data from both sides is preserved by concatenating the text (description, location, …). It remains to be seen whether that is really desirable. Also, sync statistics are slightly off: the incoming item is counted as “added” even though it gets turned into an update. * item operations: authentication problem for WebDAV when using keyring ([BMC #21311](https://bugs.meego.com/show_bug.cgi?id=21311)) The password still wasn’t looked up in the keyring when using –import/export/delete-items. * “databasePassword” source property: lookup failure in keyring ([BMC #22937](https://bugs.meego.com/show_bug.cgi?id=22937)) The databasePassword also wasn’t looked up at all when doing item operations via the command line. When configuring sources for an HTTP server, the config name typically is just the context (@foo). When using the config in the HTTP server, the config name is the peer inside that context (client@foo). Because the GNOME keyring lookup keys for the “databasePassword” (more specifically, the object name) contained the full config name which was different in both cases, looking up the saved password failed. The solution is to normalize the config name (to accomodate for different ways of spelling it) and use only the context, with @ as before. This will break existing setups where the object name in the keyring (incorrectly) includes the full config name. In that case just configure the source again to set the password anew. * Evolution Calendar: fixed detached recurrence support ([BMC #22940](https://bugs.meego.com/show_bug.cgi?id=22940)) When manipulating a meeting series with more than one detached recurrence certain sequences of operations could incorrectly fail with “UID already exists”. * iCalendar 2.0: must set VALUE in EXDATE (part of [BMC #22940](https://bugs.meego.com/show_bug.cgi?id=22940)) EXDATE has a VALUE parameter, which wasn’t defined in the XML profile. Didn’t seem to matter at all in practice, but wasn’t standard-compliant. * GTK sync-ui: wrap sync service descriptions ([BMC #7199](https://bugs.meego.com/show_bug.cgi?id=7199)) Descriptions of different sync services are not fully visible unless word-wrapping gets enabled. * source configs: don’t check “backend” unless it is needed When using a config which has sources with a backend type set which is not currently available, an error was thrown even if those sources weren’t even part of the current operation (for example, syncing another source which is currently supported). * config migration: avoid name conflicts and auto syncing of old configs ([BMC #22691](https://bugs.meego.com/show_bug.cgi?id=22691)) When (auto-)migrating a config, it was possible that a name for the peer, say foo.old, was chosen for the renamed config although there was already such a config, for example foo.old in ~/.sync4j. Besides being confusing for users, this also led to a bug in the code where it copied from the older config with the foo.old name. The main problem fixed is the disabling of auto syncing in the old config. Otherwise it was still used by syncevo-dbus-server for syncing, which triggered another auto-migration, ad infinitum… * auto syncing: must check whether enabled when looking at unknown URLs (part of [BMC #22691](https://bugs.meego.com/show_bug.cgi?id=22691)) “syncURL = insert your URL here” with “autoSync = 0” did lead to auto sync attempts although it wasn’t enabled. A check for “auto syncing enabled” was missing for the “unknown transport” case. * CalDAV/CardDAV + local storage: avoid empty properties The main motivation for this change is that a recent Apple Calendar server rejects vCards with empty BDAY property. Another reason is that keeping the data as small as possible is desirable by itself. Sending an empty property serves as a hint for the peer that the property is supported. This is not necessary when storing an item in a backend. Therefore this commit disables empty properties for all backends which do not themselves set the m_backendRule Synthesis info value. * Apple CardDAV: apply PHOTO import/export scripts by default A recent Apple Calendar server (correctly) rejects the invalid PHOTO;TYPE=unknown: property in a vCard. This internal representation must be cleared before serializing the field list. * for developers: modified backend API - ClientTestConfig modernized - InsertItemResult::m_merged turned from boolean to enum * testing and compilation changes; for example, the minimum version of libsynthesis is now checked at configure time instead of failing at runtime due to missing features in the Synthesis engine Source, Installation, Further information =============================== Source snapshots are in http://downloads.syncevolution.org/syncevolution/sources i386 and amd64 binaries for Debian-based distributions are available via the “unstable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:
deb http://downloads.syncevolution.org/apt unstable main Binaries for lpia will be made available again in the next release. These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in http://downloads.syncevolution.org/syncevolution/evolution. In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
SyncEvolution 1.1.99.6 released
- 18 August 2011
Mostly bug fixes, some improvements in testing and packaging. This release was tested successfully with DAViCal 0.9.9.4. Changes ======= * CalDAV: fixed incorrect change tracking causing “event not found” ([BMC #22329](https://bugs.meego.com/show_bug.cgi?id=22329)) * CalDAV: handle delete<->delete conflict during local sync ([BMC #22327](https://bugs.meego.com/show_bug.cgi?id=22327)) If the same event was deleted both locally and in the CalDAV server, syncing failed with “event not found”. * Google Contacts: ensure that first/middle/name are set when storing in EDS ([BMC #20864](https://bugs.meego.com/show_bug.cgi?id=20864)) Evolution and the MeeGo UX assume that first/middle/last name are set. That is not the case when a contact is created in the Google Contacts web interface. Such contacts are sent by Google without the N property. SyncEvolution now tries to recreate the name components from the FN string, by splitting at word boundaries and assuming “
deb http://downloads.syncevolution.org/apt unstable main Binaries for lpia will be made available again in the next release. These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in http://downloads.syncevolution.org/syncevolution/evolution. In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
SyncEvolution 1.1.99.5 "beyond SyncML" released
- 15 July 2011
Release 1.1.99.5 is the first release candidate for 1.2. It has gone through a long stabilization period and thus is suitable for normal users. The major new feature of the 1.2 release is support for non-SyncML protocols in general and CalDAV/CardDAV in particular. ActiveSync support is in development. Changes 1.1.1 -> 1.1.99.5 ==================== The major new feature of the 1.2 release is support for non-SyncML protocols in general and CalDAV/CardDAV in particular. ActiveSync support is in development. These protocols are implemented as backends which are combined with other backends by SyncEvolution in a so called “local sync”. The GTK sync-ui does not yet support configuring non-SyncML protocols. See the [README and man page](/documentation/syncevolution-usage) for more information on how to use the new feature via the command line. Properties not supported by SyncML servers can now be preserved locally in two-way synchronization ([BMC #15030](http://bugs.meego.com/show_bug.cgi?id=15030)). This depends on information about what properties a SyncML server supports (“CtCap”), which is typically not provided by servers. SyncEvolution contains a copy of that information for Google Contacts ([BMC #15029](http://bugs.meego.com/show_bug.cgi?id=15029)). Akonadi backend and KWallet support were merged. They are not included yet in syncevolution.org binaries. To use them compile from source. The configuration format was updated to solve a conceptual problem inherited with the legacy property names: the “type” property had multiple, sometimes conflicting roles. For example, setting the preferred data format for sync with one peer might have changed the backend selection for some other peer ([BMC #1023](http://bugs.meego.com/show_bug.cgi?id=1023)). Now “backend/databaseFormat/syncFormat/forceSyncFormat” replace “type”. “type” is still accepted by the command line as alias. Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. In contrast to earlier, more experimental releases in the 1.2 series, 1.1.99.5 and later automatically migrate configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Other changes: * a problem with enabling release mode required replacing 1.1.99.5 with a fixed 1.1.99.5a release * syncevo-http-server was enhanced considerably. See the [HTTP server HOWTO](/wiki/http-server-howto) * support NetworkManager API >= 0.9 ([BMC #19470](http://bugs.meego.com/show_bug.cgi?id=19470)) * Sync mode is recorded when running in SyncML server mode ([BMC #2786](http://bugs.meego.com/show_bug.cgi?id=2786)). * syncevo-dbus-server automatically stops when some of its libraries are updated and restarts if auto-syncing is on ([BMC #14955](http://bugs.meego.com/show_bug.cgi?id=14955)). * Using the –sync-property and –source-property command line options is optional, just specifying the property assignment is enough. * Added support for Buteo, mKCal and QtContacts in MeeGo. Buteo and mKCal were removed again from MeeGo, so the code is obsolete. The QtContacts backend may be still be useful to access items via that API, but for syncing on MeeGo the normal EDS backend is used since MeeGo reverted back to EDS as PIM storage. * code cleanup and various minor fixes/improvements, see ChangeLog Source, Installation, Further information =============================== Source snapshots are in http://downloads.syncevolution.org/syncevolution/sources i386, amd64 and lpia binaries for Debian-based distributions are available via the “unstable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:
deb http://downloads.syncevolution.org/apt unstable main These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in http://downloads.syncevolution.org/syncevolution/evolution. In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
SyncEvolution for Debian: new maintainer needed
- 30 April 2011
David Bremner, the Debian packager of SyncEvolution, [announced](http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=624540) that he is looking for a new maintainer to take over that package. If you care about SyncEvolution in Debian and Ubuntu, then please consider taking over. On this occasion let me thank David Bremner for getting SyncEvolution into Debian and for maintaining it there.
The state of syncing in open source
- 15 April 2011
There have been two blog posts recently who point out that data synchronization using open source tools still doesn’t work as well as it should: * [Adam Williamson: “The continuing state of contact + calendar synchronization suck”](http://www.happyassassin.net/2011/04/13/the-continuing-state-of-contact-calendar-synchronization-suck) * [Matěj Cepl: Synchronization sucks!](http://luther.ceplovi.cz/blog/2011/04/synchronization-sucks/) As someone who has worked hard on making synchronization suck less I can’t let this stand without commenting… in particular not when SyncEvolution is mentioned as not being reliable! ;-) Nitpicking ======= First let me point out some factual mistakes in Adam’s post: *”Maemo’s whole synchronization story”* has never been based on SyncEvolution. SyncEvolution is an add-on, supported entirely on a volunteer basis by Ove Kaveh on the device. If people do not get help for SyncEvolution in the Maemo forums, then I can’t do anything about it because I don’t have the time to keep up with everything that is said (or asked) on the web about SyncEvolution. It’s up to the Maemo community to help their peers. SyncEvolution on Linux to N900 [broken somewhere outside of SyncEvolution](https://bugs.meego.com/show_bug.cgi?id=4835) (device, Bluetooth stack) and without support by Nokia for their closed-source sync component on the device I don’t see much chances to fix it, short of some Bluetooth experts getting involved. The statements about MeeGo are also incorrect. Evolution Data Server was not the preferred PIM storage for MeeGo 1.2 until recently, so depending on it for CalDAV/CardDAV support was not an option. Adam points to [a bug report](https://bugs.meego.com/show_bug.cgi?id=319) where I captured some thoughts around the technical aspects. Perhaps this was too brief to be understand without context, but I still think that the arguments and conclusion are valid. More on that below. What exactly is the complaint? ======================= Adam tried SyncEvolution with Horde and eGroupware. Other people were able to get these combinations working, for example just a few days ago [George Runelli with eGroupware](http://www.ruinelli.ch/how-to-sync-egroupware-with-a-tablet-n900-with-syncevolution). It seems to depend a lot on the exact setup on the server side. I had offered Adam to help with diagnosing his problem with Horde (unexpected slow syncs), but he never replied to my email. I’m still convinced that the problem is not in SyncEvolution, but rather on the server side, because SyncEvolution works fine with a variety of other SyncML servers (Funambol, Memotoo, Synthesis, Mobical, to name just those that I test with nightly). It is not SyncEvolution’s fault that the open source groupwares seem to have less stable SyncML support. I tried to work with Horde and eGroupware developers a while back when I started with SyncEvolution. I had a hard time getting anyone to reply to my questions and emails, even when contacting the original developers directly. If the situation is different today, then I’d be happy to restart that effort. I’m not sure what kind of problem Matěj had with SyncEvolution. He doesn’t say in his blog post, only that it does not allow him to reliably sync with his server running Zarafa. I’m not surprised. To the best of my knowledge, the two are unable to synchronize against each other by design, because SyncEvolution is based on SyncML and Zarafa on ActiveSync. So is the complaint that SyncEvolution uses an open protocol and not a patent-encumbered proprietary protocol? Proposed solutions ============== Adam then continues to suggest that the data synchronization model itself is flawed and should be replaced with client/server model where changes are always stored on the server immediately, as in Evolution’s CalDAV and CardDAV backends. This became more clear in an email discussion after he contacted me regarding his blog post. The key difference is this: * True synchronization allows offline modification of the data. * Capable devices by design store a complete copy of the data, without depending on one particular server to remain online. Adam argued that a client/server model can be combined with caching of items and changes. But then the client/server model **becomes** synchronization and must deal with the same kind of problems that it was meant to avoid, like conflicts between items on client and server. Adam [later said](http://luther.ceplovi.cz/blog/2011/04/synchronization-sucks/#comment-37) that changes that cannot be stored anymore should simply be discarded. That doesn’t sound like a very useful approach, because users won’t be able to remember what changes might have been lost and if they do, would most likely be forced to redo them manually. PIM data is more complex then plain text, so the merge strategies that programers know how to use with source code and revision control systems do not apply. Normal users will be even more challenged. The point about not depending on a central server is important, too. In the SyncML world we have recently seen that ScheduleWorld shut down. No data was lost, because by design all users always had a full copy of their data on their own storage. The same can’t be said for all the popular Web 2.0 cloud services… My devices are not always online, for practical and economical reasons. Therefore I want the ability to make changes while offline and will continue to work on the more capable model. SyncML ====== One other aspect is the question whether the data synchronization model itself is flawed, just some protocols implementing it, or only specific implementations of these protocols. I think the approach itself is sound and useful. Adam’s own observation that other implementations of the concept seem to work better confirms that. But SyncML definitely has its flaws, both in the protocol itself and in implementations. SyncML tries to be too flexible for its own good. It allows the implementation of very dumb devices. The downside is an increased complexity on the server side. Because of its open nature, there have been a variety of implementations with varying degrees of capabilities both in the data that is supported and in the quality of the protocol implementation itself. That makes it challenging today to support the whole range of SyncML capable peers. In that sense, SyncML is a victim of its own success. The silver lining ================= I have some hopes today for CalDAV/CardDAV based synchronization. SyncEvolution 1.2 will have support for that, natively. A native implementation has the conceptual advantage that it can use meta data (resource URI + eTag) to speed up change detection, something that wouldn’t be possible when going through Evolution Data Server. CalDAV enforces that each item must have a globally unique ID, which is a considerable simplification for implementing synchronization. CardDAV unfortunately still doesn’t. Good open source implementations exist, for example Apple’s Calendar server. It passes all of the automated SyncEvolution tests. I also hear good things about DAViCal; unfortunately I haven’t found the time to test with it yet. What might be missing in both cases is good integration into a groupware solution, for those who need that.
Comment spam
- 14 March 2011
Quite a bit of spam accumulated in the site comments. Usually I deleted spam comments within a few days after receiving the comment notification email, but not all spam comments triggered those, so I missed quite a lot. I went back and reviewed all comments and deleted as necessary, so the site should be cleaner now. If I missed anything, please let me know. In order to combat spam, all comments are now held for review before being posted. This will lead to delays, so we plan to improve the anti-spam measures and remove the need to review comments manually again. If someone wants to help maintain the site, such help would be highly appreciated! It takes away time that could be spent on improving the software instead…
Question for Sony Ericsson users: charset?
- 28 February 2011
A Sony Ericsson K800 user reported a problem with non-standard characters (German umlauts in his case). It turned out that the phone uses ISO-8859-1 instead UTF-8 as encoding, without announcing that. It is possible to configure the Synthesis engine so that it does with the character set conversion correctly, but that leads to this question: *which phones need this special treatment*? I’m currently inclined to enable this for all Sony Ericsson phones, unconditionally. If you are using a Sony Ericsson a) which uses something else than ISO-8859-1 as local charset and/or b) which works correctly with non-standard characters already in SyncEvolution 1.1.1, then please leave a comment.