SyncEvolution 1.3 released#
After almost three months of public beta testing the next major version of SyncEvolution is ready for release. The pre-releases did have the desired effect of flushing out bugs not found by nightly testing alone. Thanks everyone for packaging, downloading and testing them! Project status ============== SyncEvolution synchronizes personal information management (PIM) data via various protocols (SyncML, CalDAV/CardDAV, ActiveSync). It syncs contacts, appointments, tasks and memos. It syncs to web services or to SyncML-capable phones via Bluetooth. Binaries are available for Linux desktops (using GNOME Evolution, or KDE’s Akonadi), for MeeGo and for Maemo (Nokia N900, N9). The project moved its bug tracking from bugs.meego.com to bugs.freedesktop.org. All the old issues were migrated. Please file new issues here: http://bugs.freedesktop.org/enter_bug.cgi?product=SyncEvolution The git repositories were also moved to freedesktop.org: http://cgit.freedesktop.org/SyncEvolution/ SyncEvolution 1.3 ================= New features are KDE/Akonadi and ActiveSync support, not only in the source code but also in syncevolution.org binaries. ActiveSync is the recommended way of synchronizing contacts with Google: https://syncevolution.org/wiki/google-contacts-activesync The D-Bus server and local sync were rewritten considerably, to make the code cleaner and more robust. The CalDAV backend now also supports tasks and memos. CalDAV and CardDAV can be used in combination with a SyncML peer (“bridging SyncML and WebDAV”), thus allowing a device which only supports SyncML to talk to a WebDAV service without any intermediate storage. 1.3 contains bug fixes that were not backported to 1.2.x, so upgrading is recommended. For example, SyncEvolution 1.3 is required for Evolution 3.4, otherwise photos are not exported properly. Support for Evolution >= 3.6 is in the source code, but not in syncevolution.org binaries. Further workarounds for recent changes in Google CalDAV and Funambol One Media were added. Upgrading from release 1.2.x —————————- The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts): syncevolution –configure \ syncFormat=text/x-vcard \ mobical addressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually: syncevolution –configure \ enableRefreshSync=TRUE \ funambol Upgrading from releases before 1.2 ———————————- Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Changes 1.2.2 -> 1.3 ==================== * ActiveSync: updated to work with latest activesyncd and Google, package binaries Syncing Google contacts was added to the nightly testing. Syncing contacts and events with Exchange 2012 was already working. Setup instructions and known issues are described here: https://syncevolution.org/wiki/google-contacts-activesync * phone sync: delete<->delete conflict + phone calendar+todo sync ([BMC #23744](https://bugs.meego.com/show_bug.cgi?id=23744)) When deleting an item on phone and locally, the next sync failed with ERROR messages about “object not found”. This has several reasons: - libsynthesis super data store attempts to read items which may or may not exist (triggers ERROR message) - it checks for 404 but Evolution backends only return a generic database error (causes sync to fail) * phone sync: get phone vendor and model from Device ID profile ([BMC #736](https://bugs.meego.com/show_bug.cgi?id=736)) In the past we have relied on the user-modifiable device name to be the fingerprint for matching a phone to a template which is unreliable. This release changes this in the cases where the phone supports the Device ID profile (DIP). If support for DIP is detected, then we extract the vendor and product ids and attempt to associate them with a product and vendor name by using a newly added lookup table. This lookup table has to be maintained manually and depends on contributions by users to cover more devices. See http://blixtra.org/blog/2011/09/22/syncevolution-needs-you-or-at-least-your-bluetooth-phones/ * vCalendar 1.0: fixed recurring all-day event support vCalendar 1.0 cannot represent all-day events. The workarounds for mapping iCalendar 2.0 all-day events into vCalendar 1.0 was incomplete, leading to effects like shifting EXDATEs and end times. * Funambol: ignore UID Funambol’s OneMedia sends UID, but not RECURRENCE-ID. That becomes a problem when multiple events of the same event series are added to a backend which follows the iCalendar 2.0 standard (CalDAV, EDS, KDE), because these events all look like the master event, and there can be only one of those. SyncEvolution now strips the UID from all events coming from any Funambol server (regardless of the version). If a future Funambol server release adds support for both UID and RECURRENCE-ID, then SyncEvolution will have to be updated to take advantage of the improved server. Because the RECURRENCE-ID is also getting stripped (despite not being set at the moment), SyncEvolution should continue to work as it does now even if the server changes. It would have been nice to limit this workaround to affected Funambol server versions, but an inquiry on the Funambol mailing list didn’t get a reply, therefore SyncEvolution is playing it safe and assumes that all future Funambol releases will have the same problem. * Funambol: work around PHOTO TYPE=image/jpeg A combination of Funambol Android and Funambol server recently led to the Funambol server sending PHOTO data with TYPE=image/jpeg. This is invalid and caused EDS to reject the photo (Vladimir Elisseev, “[SyncEvolution] issues with syncing photos”). Work around the problem by only keeping the part of the type after the last slash, if there is any. For image/jpeg and similar types that leads to the desired value and does not affect valid values, because those do not contain a slash (http://www.iana.org/assignments/media-types/image/index.html). * Funambol: avoid slow syncs in refresh from server libsynthesis has traditionally implemented “refresh-from-server” as “delete local data” plus “slow” sync. This is more compatible, because some servers (like Google) do not support “refresh-from-server”. But it has the downside that the server cannot know that the client won’t send any data, and Funambol’s OneMedia now only allows one slow sync before blocking the next one for a certain period of time. This is done to prevent excessive resource usage by badly behaving clients. To accomodate both kinds of servers, the new “enableRefreshSync” sync property can be set set to explicitly allow the usage of the “refresh-from-server” sync mode. It’s off by default. The Funambol template has it turned on, existing configs must be updated manually (see upgrading comments below). * Mobical (aka Everdroid): stopped testing memo syncing Memos used to work, but now only trigger an unspecific 400 error on the server side. * GTK-UI: accept service config with a username again (BMC#23106) Suppressing configs with empty username had undesired side effects: modifying configs for direct syncing with a device incorrectly triggered the same error message, without any means of entering a username. The faulty check was removed without replacement. * GTK-UI: added GTK 3 version of UI When GTK 3 is found during compilation, a GTK 3 version of the UI is built. The source code of both is different to avoid excessive use of ifdefs. At the moment, both versions offer the same features. In the long run, the GTK 3 version will replace the GTK 2 version. * command line: added refresh/one-way-from-local/remote ([BMC #23537](https://bugs.meego.com/show_bug.cgi?id=23537)) The -from-client/server sync modes are confusing because the direction of the data exchange depends on which side acts as SyncML server or client. This release introduces new modes which use -from-local/remote instead. The statistics and messages also use these variants now. The old modes are still understood, but are declared as “not recommended” in the documentation. * command line: config and source names are optional ([BMC #23783](https://bugs.meego.com/show_bug.cgi?id=23783)) The need to add “foo” and “bar” pseudo config and source names to the command line even when all parameters for the operation where explicitly specified on the command line was confusing. Now it is possible to invoke item operations without the config and source name. Names which refer to non-existent configs are still accepted, as in previous releases. Typos are handled better by producing a detailed error report which includes (as applicable): - config doesn’t exist - source doesn’t exist or not selected - backend property not set Because luids used to be positional arguments after
$ syncevolution --configure --template webdav \ syncURL=http://192.168.1.100:9000/ \ username=foo password=bar retryDuration=2s \ target-config@webdav-temp [INFO] creating configuration target-config@webdav-temp [INFO] addressbook: looking for databases... [INFO] addressbook: no database to synchronize [INFO] calendar: looking for databases... [INFO] calendar: no database to synchronize [INFO] memo: looking for databases... [INFO] memo: no database to synchronize [INFO] todo: looking for databases... [INFO] todo: no database to synchronize It timed out fairly quickly here because of the retryDuration=2s. That also gets placed in the resulting config, which is probably not desired. Aborting the operation is now supported:
$ syncevolution --configure \ --template webdav \ syncURL=http://192.168.1.100:9000/ \ username=foo password=bar \ target-config@webdav-temp [INFO] creating configuration target-config@webdav-temp [INFO] addressbook: looking for databases... ^C[INFO] Asking to suspend... [INFO] Press CTRL-C again quickly (within 2s) to stop immediately (can cause problems in the future!) ^C[INFO] Aborting immediately ... [ERROR] error code from SyncEvolution aborted on behalf of user (local, status 20017): aborting as requested by user It would be good to make the CTRL-C handling code aware that it can abort immediately instead of doing the intermediate “asking to suspend” step, which only makes sense for sync sessions. * WebDAV: support tasks and memos ([BMC #24893](https://bugs.meego.com/show_bug.cgi?id=24893)) The new backend property values “CalDAVTodo” and “CalDAVJournal” select tasks resp. memos stored in a CalDAV collection. “CalDAV” continues to select events. Events, tasks and journals can be mixed in the same resource (= URL). However, this is less efficient than storing them separately. A good CalDAV server allows filtering items by type, and SyncEvolution uses that. However, it was found that Radicale 0.7 ignores this filtering, which could have led to data loss (SyncEvolution asks for all VTODOs in preparation for a “delete all items” operation in a “CalDAVTodo” source, gets also VJOURNALs, then deletes them). Therefore SyncEvolution plays it safe and downloads the VTODO and VJOURNAL data to double-check that it is working on the right items. This causes additional traffic for well-behaving servers; currently it cannot be turned off. Tasks are exchanged as vCalendar 1.0 or iCalendar 2.0 VJOURNAL. Memos are exchanged as VTODO or plain text. The logic for storing incoming plain text is slightly different compared to the way how the EDS memo backend did it: instead of copying the first line from the text into the summary, it is now moved. In other words, the first line gets stripped. The change is primarily technically motivated; both approaches have pros and cons. * WebDAV: improved Radicale support Radicale > 0.7 will return status 200 for delete requests; is now treated like 204 by SyncEvolution. 412 ‘Preconditiona Failed’ when asking to delete an already removed item is treated like the more common 404 ‘not found’. Same with 410 ‘gone’ instead of 404 when trying to read a non-existent item. * CalDAV/CardDAV sync: improved target side output Added a “target side of local sync ready” INFO message to introduce the output which has the target context in the [INFO] tag. The sync report from the target side now has the target context embedded in brackets after the “Changes applied during synchronization” header, to avoid ambiguities. Sometimes the backend has to resend requests because of temporary issues. If the problem turned out to be permanent, there was a long period of time, retryDuration=5 minutes to be precice, in which no visible progress happened. Now SyncEvolution’s WebDAV backend will print a message like this before going to sleep until it is time to retry: [INFO @googlecalendar] operation temporarily (?) failed, going to retry in 5.0s before giving up in 18.4s: PROPFIND: Neon error code 1: 401 Unauthorized The uncertainty comes from several factors. In this example, the 401 might indicate a permanent problem (wrong credentials), or it could be Google reporting a temporary authorization problem which is (probably) meant to slow down the client while it asks the user to re-enter the password. SyncEvolution only asks for passwords once, so it tries again with the same password if it was successful with it in the past. Otherwise it gives up immediately. Another dubious example are name server lookup errors. They can be permanent (wrong host name) or temporary (name server down). SyncEvolution errs on the side of retrying, to avoid interrupting an operation which still has a chance to continue. Output from the target side of a local sync was passed through stderr redirection as chunks of text to the frontends. This had several drawbacks: - forwarding only happened when the local sync parent was processing the output redirection, which (due to limitations of the implementation) only happens when it needs to print something itself - debug messages were not forwarded - message boundaries might have been lost In particular the new INFO messages are relevant while the sync runs and need to be shown immediately. * WebDAV: –status for WebDAV source aborted The command line –status operation did not complete when applied to a CalDAV/CardDAV source. Instead it aborted because the operation took a code path where the backend was not fully initialized. * file backend: more flexible sync support for memos The databaseFormat=text/calendar for memos did not support synchronizing as plain text. When using the new databaseFormat=text/calendar+plain, vCalendar/iCalendar/plain text are all valid sync formats; the storage is iCalendar 2.0 VJOURNAL in all cases. * WebDAV: avoid potential crash during database detection When a server responds to a PROPFIND for a path with results for some other path, then SyncEvolution crashed during the search for the default calendar or address book because of a bug in the code which was meant to handle that kind of response. Apparently Yahoo Calendar did that. Now seen again in combination with Radicale 0.6.4. In general, the code was made more robust to cope with bugs in Radicale 0.6.4. Later Radicale versions fixed these issues and also worked with SyncEvolution 1.2.2 without client-side workarounds. * WebDAV: better path normalization “syncURL” and “database” properties had to end in a trailing slash, otherwise items were not found (404 errors). Now the necessary slash is added automatically. * Curl transport: support SSLServerCertificates=
[INFO] eds_contact: sent 1/1 [INFO] eds_contact: started [INFO] eds_contact: first time sync done successfully [INFO] eds_contact: starting normal sync from client (peer is server) <=== [INFO] eds_contact: started <=== [INFO] eds_contact: normal sync done successfully <=== [INFO] creating complete data backup after sync (enabled with dumpData and needed for printChanges)
Synchronization successful.
Changes applied during synchronization:
+—————|———————–|———————–|-CON-+
| | LOCAL | REMOTE | FLI |
| Source | NEW | MOD | DEL | ERR | NEW | MOD | DEL | ERR | CTS |
+—————+—–+—–+—–+—–+—–+—–+—–+—–+—–+
| eds_contact | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 |
| refresh-from-local, 2 cycles, 0 KB sent by client, 0 KB received |
^^^^^^^^
| item(s) in database backup: 1 before sync, 1 after it |
+—————+—–+—–+—–+—–+—–+—–+—–+—–+—–+
| start Tue Feb 7 17:07:49 2012, duration 0:03min |
| synchronization completed successfully |
+—————+—–+—–+—–+—–+—–+—–+—–+—–+—–+
* SyncEvolution <-> SyncEvolution sync: negotiate UID support via SyncCap (\[BMC #22783\](https://bugs.meego.com/show\_bug.cgi?id=22783)) The semantic of UID/RECURRENCE-ID in calendar data is now tracked per data store involved in a sync. If full iCalendar 2.0 semantic (= IDs are globally unique) is guaranteed, then pairs are found based on these IDs. Otherwise pairs must be found by looking at item attributes. Previously a hack was used to detect this kind of support (any kind of SyncEvolution instance was assumed to support it, although some backends do not). \* engine: add DTSTAMP+LAST-MODIFIED before writing calendar items When writing calendar items into a backend storage as iCalendar 2.0 or vCalendar 1.0, they should have DTSTAMP and LAST-MODIFIED values. DTSTAMP is expected by some CalDAV servers (like Apple). LAST-MODIFIED is usually added by the storage, but not always. In the text/plain -> syncevolution -> text/calendar -> Radicale -> EDS -> syncevolution chain the LAST-MODIFIED was not added by Radicale, which caused problems for change tracking in an EDS-based SyncEvolution. Also necessary when importing from a phone using vCalendar without DTSTAMP directly into CalDAV. \* autotools: ensure that link lines are complete As mentioned by Tino Keitel on the mailing list, some libs and executables were only implicitly linked against libraries that they called directly. This happened to work by chance because these libraries ended up in the running executable anyway, due to indirect loading. Now there is a "make installcheck" test for this kind of defect and the makefiles were updated to avoid it. One exception is libsmltk, which depends on the caller providing SySync logging support. \* syncevolution.org packages: fixed D-Bus server autostart in .deb and .rpm packages syncevo-dbus-server wasn't started automatically as part of a user session because /etc/xdg/autostart/syncevo-dbus-server.desktop wasn't included in the packages. This broke auto syncing after a session restart (required manually starting SyncEvolution). \* syncevolution.org packages: support KDE The traditional "syncevolution-evolution" package was replaced with "syncevolution-bundle". A meta "syncevolution-evolution" package depends on it, to support seamless updates for users who have "syncevolution-evolution" installed. Binary dependencies of the main .deb are ignored for backends because loading them is optional. The new "syncevolution-kde" package has the right dependencies for KDE/Akonadi, while "syncevolution-evolution" mostly just lists standard libs if the "EDS compatibility" mode is used, where libebook/libecal are loaded dynamically. Platform specific code (GNOME keyring, KDE wallet) was moved into loadable, optional modules, to allow installation of the SyncEvolution bundle without forcing the installation of unused system components. \* D-Bus: use GIO D-Bus instead of libdbus if available When compiling from source, the more modern GIO D-Bus is used instead of libdbus if available and recent enough (>= 2.30). syncevolution.org binaries still use libdbus, to stay compatible with older Linux distros. \* several minor bug fixes syncevo-dbus-server now runs under valgrind in the nightly testing, plus several more test scenarios were added. This helped to find and fix various minor memory handling issues. \* developers: backend API changes beginSync/endSync() (aka m\_startDataRead/m\_endDataWrite) may now be called multiple times per SyncSource instance life cycle. SyncSources derived from TrackingSyncSource should work without changes. Use the Client::Source::\*::testChangesMultiCycles test to check whether your backend supports this correctly. Reading and deleting must throw a 404 status exception when an item is not found. The Client::Source::\*::\*404 tests cover this. The special semantic of the former RegisterSyncSource::InactiveSource (invalid pointer of value 1) caused bugs, like using it in --print-databases (=> segfault) or not being able to store the result of a createSource() directly in a smart pointer (=> potential leak in SyncSource::createSource()). Obviously a bad idea to start with. Replaced with a RegisterSyncSource::InactiveSource() method which creates a real, inactive SyncSource instance which can and must be deleted by the caller. This is a SyncSource API change for backend developers. Instead of RegisterSyncSource::InactiveSource, return RegisterSyncSource::InactiveSource(). Comparisons against RegisterSyncSource::InactiveSource needs to be replaced with a call to the new SyncSource::isInactive(). Long-running backend calls are encouraged to check for events on the main glib context (either in a loop or with g\_main\_context\_iteration(NULL)) and abort when SuspendFlags::getSuspendFlags().getState() returns SuspendFlags::ABORT. Implementing the improved local sync output required extending the D-Bus API. The Server.LogOutput signal now has an additional "process name" parameter. Normally it is empty. For messages originating from the target side, it carries that extra target context string. This D-Bus API change is backward compatible. Older clients can still subscribe to and decode the LogOutput messages, they'll simply ignore the extra parameter. Newer clients expecting that extra parameter won't work with an older D-Bus daemon: they'll fail to decode the D-Bus message. \* packagers: libgdbussyncevo is now installed as a normal library in /usr/lib, even though SyncEvolution is the only user. pcrecpp is now a new hard dependency. Changes 1.2.99.4 -> 1.3 ======================= \* D-Bus server + GIO D-Bus: shutdown fix When compiled against GIO D-Bus (not the case in syncevolution.org binaries), the syncevo-dbus-server occasionally shut down before sending out all pending D-Bus messages. Showed up only in nightly testing. \* D-Bus server + GIO D-Bus: fix auto-activation (Debian bug #599247) When syncevo-dbus-server was started on demand by the D-Bus daemon, then it registered itself with the daemon before it was ready to serve requests. Only happened in combination with GIO D-Bus and thus was not a problem before 1.2.99.x. One user-visible effect was that the GTK UI did not select the default service when it was started for the first time, because it could not retrieve that information from syncevo-dbus-server. \* local sync: fix timeout with local sync with libdbus When using libdbus instead of GIO D-Bus (as done by syncevolution.org binaries and SyncEvolution on Maemo), local sync may have aborted after 25 seconds when syncing many items with a D-Bus timeout error: \[ERROR] sending message to child failed: org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible ca Reported by Toke Høiland-Jørgensen for Harmattan. Somehow not encountered elsewhere. \* KDE: check for D-Bus to avoid crash in KApplication ([BMC #25596\](https://bugs.meego.com/show_bug.cgi?id=25596)) Some unnamed version of KDE crashes in KApplication when invoked without a D-Bus session. The reporter ran into this when compiling from source, because the SyncEvolution binary is invoked as part of the build process, which ran outside of a D-Bus session. Avoid the crash by checking for a D-Bus session bus before instantiating KApplication. Instantiating KApplication was added for KWallet support. Without D-Bus, KWallet does not work either, therefore throw an explicit error when the lack of D-Bus is detected. \* Funambol: work around PHOTO TYPE=image/jpeg A combination of Funambol Android and Funambol server recently led to the Funambol server sending PHOTO data with TYPE=image/jpeg. This is invalid and caused EDS to reject the photo (Vladimir Elisseev, "\[SyncEvolution] issues with syncing photos"). Work around the problem by only keeping the part of the type after the last slash, if there is any. For image/jpeg and similar types that leads to the desired value and does not affect valid values, because those do not contain a slash (http://www.iana.org/assignments/media-types/image/index.html). \* syncevo-http-server: fixed printing of server debug output Python failed to call logSyncEvoOutput() after adding the additional 'process' parameter to LogOutput because it extracts all four parameters and then cannot pass them to logSyncEvoOutput(). Now logSyncEvoOutput() uses the new process information to instantiate a logger with the right prefix, using 'sync' as fallback for messages without that information (as before). \* Some minor code and test cleanup. Source, Installation, Further information ========================================= Source code bundles for users are available in http://downloads.syncevolution.org/syncevolution/sources and the original source is in [the git repositories\](http://cgit.freedesktop.org/SyncEvolution/). i386, lpia and amd64 binaries for Debian-based distributions are available via the "stable" syncevolution.org repository. Add the following entry to your /apt/source.list:
deb http://downloads.syncevolution.org/apt stable main
` Then install “syncevolution-evolution”, “syncevolution-kde” and/or “syncevolution-activesync”. These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy), except for “syncevolution-activesync” which depends on libraries in Debian Squeeze, for example EDS 3.4. Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/). In contrast to 0.8.x archives, the 1.x .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.