SyncEvolution synchronizes personal information management (PIM) data via
various protocols (SyncML, CalDAV/CardDAV, ActiveSync). It syncs contacts,
appointments, tasks and memos. It syncs to web services or to SyncML-capable
phones via Bluetooth.
See the documentation to read an overview, to install SyncEvolution, or to
configure and run it. SyncEvolution is stable and works. The compatibility page
explains what platforms SyncEvolution runs on and which servers it works
with. Known limitations are documented there as well as in the list of known
issues. If you need help, then please let us know.
SyncEvolution was started in 2006 by Patrick Ohly with a goal of writing a
small, reliable solution for Evolution PIM data synchronization without
reinventing the wheel. During SyncEvolution’s development, special attention
was paid to automated testing and coverage of corner cases of the SyncML, vCard
and iCalendar standards to ensure that no data gets lost or mangled. Originally
it used the Funambol C++ client library as its SyncML engine, which was already
available as open source at that time for Windows and Windows Mobile. It was
ported to Linux as part of the SyncEvolution development. Since then the
project has grown considerably. In 2009, it was integrated into Moblin and it
switched to the Synthesis SyncML engine, with the goal of adding direct
synchronization with phones. In 2010 release 1.0 delivered that feature, both
in MeeGo 1.0 and on Linux desktops.
This release modernizes the code base and removes usage of out-dated
libraries and APIs. All Python scripts require Python 3. The major
version bump reflects that this release is not just a minor bug fix.
However, no new features were added.
Binaries on syncevolution.org get built for distros >= Ubuntu Bionic
and Debian Buster. Testing is now based on Docker containers instead
of a custom schroot solution, so adding testing against other distros
will be easier in the future. Compilation on Fedora Rawhide was
already added.
Maintenance release. syncevolution.org binaries are now getting
compiled for distros >= Ubuntu Xenial 16.04 LTS. Usage of deprecated
libraries (GNOME keyring) and APIs (SoupAsyncSession) was
replaced. libical v3 is supported.
The code now compiles more cleanly with recent compilers and depends
on C++11 support.
Maintenance release. syncevolution.org binaries are now getting compiled for distros >= Ubuntu Trusty 14.04 LTS, which allowed removing several hacks that were needed when building binaries that also had to run on older distros. Compilation from source for old distros should still work as before, but is not getting tested anymore. Compile problems with recent libraries (libical v2) and tools (GCC v6) were resolved. Syncing via Bluetooth with certain phones now should work reliably in incremental mode. New backends for the Trinity Desktop Environment (TDE) were added to the source code. Details: * ObexTransportAgent.cpp: properly shut down connection (FDO #91485) Apparently there’s a race condition in the OBEX transport that causes the connection to phones via Bluetooth to be shut down prematurely. Some phones react by doing a slow sync instead of an incremental sync the next time. * support non-readable parent directories (FDO #91000) The previous mkdir_p() walked down top to bottom and checked each path entry as it went along. That approach failed unnecessarily when some existing parent directory could not be read (non-readable /home, for example). * avoid using dbus-launch (Debian #836399) dbus-launch is considered deprecated because of the X11 dependency. See https://lists.debian.org/debian-devel/2016/08/msg00554.html “Mass bug filing: use and misuse of dbus-launch (dbus-x11)” The dbus-session.sh script still needs to start the D-Bus daemon when used in the nightly testing, so the code now does it by invoking the dbus-daemon directly. syncevo-http-server still has some usage of dbus-launch left, but that’s strictly for systems which don’t have the more modern D-Bus. * syncevo-dbus-server integrates better with systemd (FDO #92164) A .service file allows the D-Bus daemon to start the service via systemd, thus ensuring that the process environment is correct. Patch from Simon McVittie. Auto-starting as part of the desktop login uses D-Bus activation if the “dbus-send” tool is installed. * syncevolution.org: compile on Ubuntu Trusty, libical v1/v2 compatibility syncevolution.org binaries are now getting compiled on Ubuntu Trusty and thus no longer support distros with older EDS. The code should still compile against older EDS (for example, for Maemo), but that is not getting tested anymore. This allows removing the dynamic linker hacks related to older libraries, which was only used in those binaries. Instead, backends using libical or EDS get compiled on Ubuntu Trusty and then the soname of those libs get patched to make the backend module usable in combination with a different set of libs. That patching is part of a script maintained in the syncevolution.org build infrastructure. This approach was already used before to generate different EDS backends for EDS versions with the newer EClient API, because that turned out to be easier than the dynamic loading approach. It works because none of the methods used by SyncEvolution changed their ABI, only some other parts of the libraries did. Should there ever be a situation again that cannot be handled like this, then backends also get compiled on different distros than Ubuntu Trusty (for example, the ActiveSync backend for Debian Stretch is built on Debian Stretch). libical still requires one special hack: system time zone loading in libical v1 (and only in that version, v2 has builtin support again) must be overridden such that time zones are generated with rules instead of transitions because that is more compatible with the peers that SyncEvolution exchanges data with. That hack now relies on overriding the two relevant functions inside the main binaries (has to be there, otherwise libical still ends up calling its own internal implementation). The overriding code is in libsyncevo-icaltz-util.so.0 and depends on libical.so.1. If libsyncevo-icaltz-util.so.0 can be loaded, the wrappers in the main binary use it, otherwise they fall through to the code from the current libical.so, which then should be libical.so.2 or more recent. This hack is active by default when libical v1 is detected during configuration. * optionally show debug output in –version output SYNCEVOLUTION_DEBUG=1 syncevolution –daemon=no –version now dumps also the debug information gathered by the binary compatibility code. It was only available in sync logs before. * various build fixes for libical v2, GCC v6/C++14 Upgrading from releases <= 1.3.99.4: ———————————— If the value of “username/databaseUser/proxyUser” contains a colon, the “user:” prefix must be added to the value, to continue treating it like a plain user name and not some reference to an unknown identity provider (like “id:”, “goa:”, “signon:”, etc.). The lookup of passwords in GNOME Keyring was updated slightly in 1.3.99.5. It may be necessary to set passwords anew if the old one is no longer found. Upgrading from release 1.2.x: —————————– The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts): syncevolution –configure \ syncFormat=text/x-vcard \ mobical addressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually: syncevolution –configure \ enableRefreshSync=TRUE \ funambol Upgrading from releases before 1.2: ———————————– Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Source, Installation, Further information ========================================= Source code bundles for users are available in https://download.01.org/syncevolution/syncevolution/sources and the original source is in the git repositories http://cgit.freedesktop.org/SyncEvolution/ i386 and amd64 binaries for Debian-based distributions are available via the “stable” syncevolution.org repository. Add the following entry to your /etc/apt/source.list:
The GPG key for the repository needs to be imported as root with:
Maintenance release. Binaries now also get compiled for Debian 8.0 “Jessie”. The apt repositories on syncevolution.org (mirrored on download.01.org) are signed with a different key than before, see “Installation” below. Details: * avoid time zone issue with Funambol server The Funambol iCalendar 2.0 parser fails to handle time zones with quotation marks around the TZID value, which is something that SyncEvolution started to add in 1.4.99.3. While it is valid to quote like that, it is not necessary, so avoid quoting in this case to restore interoperability. * syncevo-http-server: stop using deprecated twisted.web.error (FDO #90419) This has become a real problem for example on Fedora 22 where the old name is no longer available. * syncevo-http-server: use TLS instead of SSLv3 This fixes a potential security risk and connection problems with clients that don’t support SSLv3 anymore. * syncing: avoid segfault for invalid text inside items (FDO #90118) As reported by Canonical, syncing fails if data items contain text which is not correct UTF-8 in one of the fields that SyncEvolution logs in the command line output (like SUMMARY of a calendar event). That is because the byte string coming from the item is passed unchecked to the D-Bus implementation for transmission via D-Bus. But D-Bus strings must be correct UTF-8, so depending on the D-Bus library in use, one gets a segfault (GIO D-Bus, due to an unchecked NULL pointer access) or an “out of memory” error (libdbus, which checks for NULL). SyncEvolution now replaces invalid bytes with a question mark in its output while preserving the rest of the text. * file backend: log item manipulation Extracting a meaningful description of each item from the Synthesis engine when updating and adding items is easy to do for items of certain known types (contacts and calendar items). * command line: preserve log prefix of target side of local sync In some cases, the prefix which was supposed to be embedded in the log messages from the target side of a local sync got lost on the way to the command line tool. Primarily this affected the added/updated/deleted messages, as in: [INFO remote@client] @client/addressbook: started [INFO remote@client] updating “Joan Doe” [INFO remote@client] @client/addressbook: received 1/1 * compile fix: use ${PKG_CONFIG} instead of pkg-config. This fixes the build on Exherbo that only has prefixed versions of pkg-config. * WebDAV: handle 403 during Google OAuth authentication When sending an access token with insufficient scope (for example, because the Ubuntu Online Accounts service definition was incomplete, as documented in FDO #86824), Google responds with a 403 “service denied” error. This is now dealt with by retrying, just as for a transient 401 error. * CalDAV: more efficient “is empty” check (FDO #86335) Since 1.4.99.4, syncing WebDAV collections always checks first whether there are items in the collections. This was partly done for slow sync prevention (which is not necessary for empty collections), partly for the “is the datastore usable” check. However, this did not take into account that for CalDAV collections, the entire content gets downloaded for this check. That is because filtering by item type (VEVENT vs. VJOURNAL) is not implemented correctly by all servers. So now all CalDAV syncs, whether incremental or slow, always transfered all items, which is not the intention (incremental syncs should be fast and efficient). This release adds a more efficient isEmpty() check: for simple CardDAV collections, only luid and etag get transferred, as in listAllItems(). This is the behavior from 1.5. For CalDAV, a report with a filter for the content type is used and the transfer gets aborted after the first item, without actually double-checking the content of the item. This is different from listAllItems(), which really transfers the content. This extra content check would only be needed for some old servers (Radical 0.7) and is not essential, because reporting “not empty” even when empty is safe. * WebDAV: send Basic Auth via http in some cases (FDO #57248) It turned out that finding databases on an Apple Calendar server accessed via http depends on sending Basic Auth even when the server does not ask for it: without authentication, there is no information about the current principal, which is necessary for finding the user’s databases. To make this work again, sending the authentication header is now forced for plain http if (and only if) the request which should have returned the principal URL fails to include it. This implies sending the same request twice, but as this scenario should be rare in practise (was only done for testing), this is acceptable. * Ubuntu Online Accounts: support plain text credentials The backend for UOA was rewritten by Alberto Mardegan and now also can use plain username/password credentials stored in UOA. * various compiler error and warning fixes Upgrading from releases <= 1.3.99.4: ———————————— If the value of “username/databaseUser/proxyUser” contains a colon, the “user:” prefix must be added to the value, to continue treating it like a plain user name and not some reference to an unknown identity provider (like “id:”, “goa:”, “signon:”, etc.). The lookup of passwords in GNOME Keyring was updated slightly in 1.3.99.5. It may be necessary to set passwords anew if the old one is no longer found. Upgrading from release 1.2.x: —————————– The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts): syncevolution –configure \ syncFormat=text/x-vcard \ mobical addressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually: syncevolution –configure \ enableRefreshSync=TRUE \ funambol Upgrading from releases before 1.2: ———————————– Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Source, Installation, Further information ========================================= Source code bundles for users are available in https://download.01.org/syncevolution/syncevolution/sources and the original source is in [the git repositories](http://cgit.freedesktop.org/SyncEvolution/). i386, lpia and amd64 binaries for Debian-based distributions are available via the “stable” syncevolution.org repository. Add the following entry to your /apt/source.list:
debhttps://download.01.org/syncevolution/aptstablemain The GPG key for the repository needs to be imported as root with:
apt-keyadv--keyserverkeyserver.ubuntu.com--recv-keysB2EC3981 Then install “syncevolution-evolution”, “syncevolution-kde” and/or “syncevolution-activesync”. These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 10.04 LTS (Lucid), except for ActiveSync binaries which were compiled for Debian Wheezy, Ubuntu Saucy and Ubuntu Trusty, and Debian Jessie. The packages mentioned above are meta-packages which pull in suitable packages matching the distro during installation. Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](https://download.01.org/syncevolution/syncevolution/). In contrast to 0.8.x archives, the 1.x .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps. More specific [HOWTOs](/wiki/howto) can be found in the Wiki.
About 1.5 ========= This version is the new stable, supported version of SyncEvolution. Compared to the last 1.4.99.4 pre-release only minor changes were done (see below). This section summarizes the changes since the last stable release, 1.4.1. Based on community feedback and discussions, the terminology used in SyncEvolution for configuration, local sync and database access was revised. Some usability issues with setting up access to databases were addressed. Interoperability with WebDAV servers and in particular Google Contacts was enhanced considerably. Access to iCloud contacts was reported as working when using username=foobar@icloud.com and password, but is not formally tested. Syncing with iCloud calendars ran into a server limitation (reported as 17001498 “CalDAV REPORT drops calendar data”) and needs further work ([FDO #72133](https://bugs.freedesktop.org/show_bug.cgi?id=72133)). Contact data gets converted to and from the format typically used by CardDAV servers, so now anniversary, spouse, manager, assistant and instant message information are exchanged properly. Custom labels get stored in EDS as extensions and no longer get lost when updating some other aspects of a contact. However, Evolution does not show custom labels and removes them when editing a property which has a custom label ([BGO #730636](https://bugzilla.gnome.org/show_bug.cgi?id=730636)). Scanning for CardDAV/CalDAV resources was enhanced. It now finds additional calendars with Google CalDAV. For Google, the obsolete SyncML config template was removed and CalDAV/CardDAV were merged into a single “Google” template. Using Google Calendar/Contacts with OAuth2 authentication on a headless server becomes a bit easier: it is possible to set up access on one system with a GUI using either gSSO or GNOME Online Accounts, then take the OAuth2 refresh token and use it in SyncEvolution on a different system. See [the oauth2 backend README](http://cgit.freedesktop.org/SyncEvolution/syncevolution/tree/src/backends/oauth2/README) for details. syncevolution.org binaries do not include this feature. The PIM Manager API also supports Google Contact syncing. Some problems with suspending a PBAP sync were fixed. Suspend/abort can be tested with the sync.py example. Performance is better for local syncs and PBAP caching. The most common case, a two-way sync with no changes on either side, no longer rewrites any meta data files. CPU consumption during local sync was reduced to one third by exchanging messages via shared memory instead of internal D-Bus. Redundant vCard decode/encode on the sending side of PBAP and too agressive flushing of meta data during a normal sync were removed. The EDS memo backend is able to switch between syncing in plain text and iCalendar 2.0 VJOURNAL automatically. Graham Cobb fixed some all-day conversion issues in activesyncd. The updated version is part of the 1.5 release on syncevolution.org. Details: * source -> datastore rename, improved terminology The word “source” implies reading, while in fact access is read/write. “datastore” avoids that misconception. Writing it in one word emphasizes that it is single entity. While renaming, also remove references to explicit –*-property parameters. The only necessary use today is “–sync-property ?” and “–datastore-property ?”. –datastore-property was used instead of the short –store-property because “store” might be mistaken for the verb. It doesn’t matter that it is longer because it doesn’t get typed often. –source-property must remain valid for backward compatility. As many user-visible instances of “source” as possible got replaced in text strings by the newer term “datastore”. Debug messages were left unchanged unless some regex happened to match it. The source code will continue to use the old variable and class names based on “source”. Various documentation enhancements: Better explain what local sync is and how it involves two sync configs. “originating config” gets introduces instead of just “sync config”. Better explain the relationship between contexts, sync configs, and source configs (“a sync config can use the datastore configs in the same context”). An entire section on config properties in the terminology section. “item” added (Todd Wilson correctly pointed out that it was missing). Less focus on conflict resolution, as suggested by Graham Cobb. Fix examples that became invalid when fixing the password storage/lookup mechanism for GNOME keyring in 1.4. The “command line conventions”, “Synchronization beyond SyncML” and “CalDAV and CardDAV” sections were updated. It’s possible that the other sections also contain slightly incorrect usage of the terminology or are simply out-dated. * local sync: allow config name in syncURL=local:// Previously, only syncURL=local://@ was allowed and used the “target-config@context name” config as target side in the local sync. Now “local://config-name@context-name” or simply “local://config-name” are also allowed. “target-config” is still the fallback if only a context is give. It also has one more special meaning: “–configure target-config@google” will pick the “Google” template automatically because it knows that the intention is to configure the target side of a local sync. It does not know that when using some other name for the config, in which case the template (if needed) must be specified explicitly. The process name in output from the target side now also includes the configuration name if it is not the default “target-config”. * command line: revise usability checking of datastores When configuring a new sync config, the command line checks whether a datastore is usable before enabling it. If no datastores were listed explicitly, only the usable ones get enabled. If unusable datastores were explicitly listed, the entire configure operation fails. This check was based on listing databases, which turned out to be too unspecific for the WebDAV backend: when “database” was set to some URL which is good enough to list databases, but not a database URL itself, the sources where configured with that bad URL. Now a new SyncSource::isUsable() operation is used, which by default just falls back to calling the existing Operations::m_isEmpty. In practice, all sources either check their config in open() or the m_isEmpty operation, so the source is usable if no error is enountered. For WebDAV, the usability check is skipped because it would require contacting a remote server, which is both confusing (why does a local configure operation need the server?) and could fail even for valid configs (server temporarily down). The check was incomplete anyway because listing databases gave a fixed help text response when no credentials were given. For usability checking that should have resulted in “not usable” and didn’t. The output during the check was confusing: it always said “listing databases” without giving a reason why that was done. The intention was to give some feedback while a potentially expensive operation ran. Now the isUsable() method itself prints “checking usability” if (and only if!) such a check is really done. Sometimes datastores were checked even when they were about to be configure as “disabled” already. Now checking such datastores is skipped. * command line: fix –update from directory The “–update
This is the first release candidate for 1.5. No further changes are planned except for fixing yet-to-be-discovered bugs - so find them now! :-) One focus in this release was on minimizing CPU consumption and disk writes. The most common case, a two-way sync with no changes on either side, no longer rewrites any meta data files. CPU consumption during local sync was reduced to one third by exchanging messages via shared memory instead of internal D-Bus. Redundant vCard decode/encode on the sending side of PBAP and too agressive flushing of meta data during a normal sync were removed. Altogether, sending 1000 contacts with photo data in a refresh-from-server local sync takes only one sixth of the CPU cycles compared to 1.3.99.3 (measured with valgrind’s callgrind on x86_64). Based on community feedback and discussions, the terminology used in SyncEvolution for configuration, local sync and database access was revised. Some usability issues with setting up access to databases were addressed. For Google, the obsolete SyncML config template was removed and CalDAV/CardDAV were merged into a single “Google” template. Using Google Calendar/Contacts with OAuth2 authentication on a headless server becomes a bit easier: it is possible to set up access on one system with a GUI using either gSSO or GNOME Online Accounts, then take the OAuth2 refresh token and use it in SyncEvolution on a different system. See the [new OAuth2 backend README](http://cgit.freedesktop.org/SyncEvolution/syncevolution/tree/src/backends/oauth2/README). Some issues accessing Apple iCloud were fixed such that CardDAV works by just giving SyncEvolution username=foobar@icloud.com and password. No throrough testing was done, so iCloud support is still experimental. The PIM Manager API also supports Google Contact syncing. Some problems with suspending a PBAP sync were fixed. Suspend/abort can be tested with the sync.py example. The EDS memo backend is able to switch between syncing in plain text and iCalendar 2.0 VJOURNAL automatically. Details: ——— * oauth2: new backend using libsoup/libcurl New backend implements identity provider for obtaining OAuth2 access token for systems without HMI support. Access token is obtained by making direct HTTP request to OAuth2 server and using refresh token obtained by user in some other way. New provider automatically updates stored refresh token when OAuth2 server is issuing new one. * PBAP: use raw text items This avoids the redundant parse/generate step on the sending side of the PBAP sync. * datatypes: raw text items with minimal conversion ([FDO #52791](https://bugs.freedesktop.org/show_bug.cgi?id=52791)) When using “raw/text/calendar” or “raw/text/vcard” as SyncEvolution “databaseFormat”, all parsing and conversion is skipped. The backend’s data is identical to the item data in the engine. Finding duplicates in a slow sync is very limited when using these types because the entire item data must match exactly. This is useful for the file backend when the goal is to store an exact copy of what a peer has or for limited, read-only backends (PBAP). The downside of using the raw types is that the peer is not given accurate information about which vCard or iCalendar properties are supported, which may cause some peers to not send all data. * engine: flush map items less frequently The Synthesis API does not say explicitly, but in practice all map items get updated in a tight loop. Rewriting the m_mappingNode (case insensitive string comparisons) and serialization to disk (std::ostrstream) consume a significant amount of CPU cycles and cause extra disk writes that can be avoided by making some assumptions about the sequence of API calls and flushing only once. * SoupTransport: drop CA file check It used to be necessary to specify a CA file for libsoup to enable SSL certificate checking. Nowadays libsoup uses the default CA store unless told otherwise, so the check in SyncEvolution became obsolete. However, now there is a certain risk that no SSL checking is done although the user asked for it (when libsoup is not recent enough or compiled correctly). * local sync: exchange SyncML messages via shared memory Encoding/decoding of the uint8_t array in D-Bus took a surprisingly large amount of CPU cycles relative to the rest of the SyncML message processing. Now the actual data resides in memory-mapped temporary files and the D-Bus messages only contain offset and size inside these files. Both sides use memory mapping to read and write directly. For caching 1000 contacts with photos on a fast laptop, total sync time roughly drops from 6s to 3s. To eliminate memory copies, memory handling in libsynthesis or rather, libsmltk is tweaked such that it allocates the buffer used for SyncML message data in the shared memory buffer directly. This relies on knowledge of libsmltk internals, but those shouldn’t change and if they do, SyncEvolution will notice (“unexpected send buffer”). * local sync: avoid updating meta data when nothing changed The sync meta data (sync anchors, client change log) get updated after a sync even if nothing changed and the existing meta data could be used again. This can be skipped for local sync, because then SyncEvolution can ensure that both sides skip updating the meta data. With a remote SyncML server that is not possible and thus SyncEvolution has to update its data. This optimization is only used for local syncs with one source. It is based on the observation that when the server side calls SaveAdminData, the client has sent its last message and the sync is complete. At that point, SyncEvolution can check whether anything has changed and if not, skip saving the server’s admin data and stop the sync without sending the real reply to the client. Instead the client gets an empty message with “quitsync” as content type. Then it takes shortcuts to close down without finalizing the sync engine, because that would trigger writing of meta data changes. The server continues its shutdown normally. This optimization is limited to syncs with a single source, because the assumption about when aborting is possible is harder to verify when multiple sources are involved. * PIM: include CardDAV in CreatePeer() This adds “protocol: CardDAV” as a valid value, with corresponding changes to the interpretation of some existing properties and some new ones. The API itself is not changed. Suspending a CardDAV sync is possible. This freezes the internal SyncML message exchange, so data exchange with the CardDAV server may continue for a while after SuspendPeer(). Photo data is always downloaded immediately. The “pbap-sync” flag in SyncPeerWithFlags() has no effect. Syncing can be configured to be one-way (local side is read-only cache) or two-way (local side is read/write). Meta data must be written either way, to speed up caching or allow two-way syncing. The most common case (no changes on either side) will have to be optimized such that existing meta data is not touched and thus no disk writes occur. * PIM: handle SuspendPeer() before and after transfer ([FDO #82863](https://bugs.freedesktop.org/show_bug.cgi?id=82863)) A SuspendPeer() only succeeded while the underlying Bluetooth transfer was active. Outside of that, Bluez errors caused SyncEvolution to attempt a cancelation of the transfer and stopped the sync. When the transfer was still queueing, obexd returns org.bluez.obex.Error.NotInProgress. This is difficult to handle for SyncEvolution: it cannot prevent the transfer from starting and has to let it become active before it can suspend the transfer. Canceling would lead to difficult to handle error cases (like partially parsed data) and therefore is not done. The Bluez team was asked to implement suspending of queued transfers (see “org.bluez.obex.Transfer1 Suspend/Resume in queued state” on linux-bluetooth@vger.kernel.org), so this case might not happen anymore with future Bluez. When the transfer completes before obexd processes the Suspend(), org.freedesktop.DBus.Error.UnknownObject gets returned by obexd. SyncEvolution can ignore errors which occur after the active transfer completed. In addition, it should prevent starting the next one. This may be relevant for transfer in chunks, although the sync engine will also stop asking for data and thus typically no new transfer gets triggered anyway. * PIM: add suspend/resume/abort to sync.py CTRL-C while waiting for the end of a sync causes an interactive prompt to appear where one can choose been suspend/resume/abort and continuing to wait. CTRL-C again in the prompt aborts the script. * PIM: fix sync.py –sync-flags The help text used single quotes for the JSON example instead of the required double quotes. Running without –sync-flags was broken because of trying to parse the empty string as JSON. * command line: revise usability checking of datastores When configuring a new sync config, the command line checks whether a datastore is usable before enabling it. If no datastores were listed explicitly, only the usable ones get enabled. If unusable datastores were explicitly listed, the entire configure operation fails. This check was based on listing databases, which turned out to be too unspecific for the WebDAV backend: when “database” was set to some URL which is good enough to list databases, but not a database URL itself, the sources where configured with that bad URL. Now a new SyncSource::isUsable() operation is used, which by default just falls back to calling the existing Operations::m_isEmpty. In practice, all sources either check their config in open() or the m_isEmpty operation, so the source is usable if no error is enountered. For WebDAV, the usability check is skipped because it would require contacting a remote server, which is both confusing (why does a local configure operation need the server?) and could fail even for valid configs (server temporarily down). The check was incomplete anyway because listing databases gave a fixed help text response when no credentials were given. For usability checking that should have resulted in “not usable” and didn’t. The output during the check was confusing: it always said “listing databases” without giving a reason why that was done. The intention was to give some feedback while a potentially expensive operation ran. Now the isUsable() method itself prints “checking usability” if (and only if!) such a check is really done. Sometimes datastores were checked even when they were about to be configure as “disabled” already. Now checking such datastores is skipped. * EDS: memo syncing as iCalendar 2.0 ([FDO #52714](https://bugs.freedesktop.org/show_bug.cgi?id=52714)) When syncing memos with a peer which also supports iCalendar 2.0 as data format, the engine will now pick iCalendar 2.0 instead of converting to/from plain text. The advantage is that some additional properties like start date and categories can also be synchronized. The code is a lot simpler, too, because the EDS specific iCalendar 2.0 <-> text conversion code can be removed. * datatypes: text/calendar+plain revised heuristic When sending a VEVENT, DESCRIPTION was set to the SUMMARY if empty. This may have been necessary for some peers, but for notes (= VJOURNAL) we don’t know that (hasn’t been used in the past) and don’t want to alter the item unnecessarily, so skip that part and allow empty DESCRIPTION. When receiving a plain text note, the “text/calendar+plain” type used to store the first line as summary and the rest as description. This may be correct in some cases and wrong in others. The EDS backend implemented a different heuristic: there the first line is copied into the summary and stays in the description. This makes a bit more sense (the description alone is always enough to understand the note). Therefore and to avoid behavioral changes for EDS users when switching the EDS backend to use text/calendar+plain, the engine now uses the same approach. * source -> datastore rename, improved terminology The word “source” implies reading, while in fact access is read/write. “datastore” avoids that misconception. Writing it in one word emphasizes that it is single entity. While renaming, also remove references to explicit –*-property parameters. The only necessary use today is “–sync-property ?” and “–datastore-property ?”. –datastore-property was used instead of the short –store-property because “store” might be mistaken for the verb. It doesn’t matter that it is longer because it doesn’t get typed often. –source-property must remain valid for backward compatility. As many user-visible instances of “source” as possible got replaced in text strings by the newer term “datastore”. Debug messages were left unchanged unless some regex happened to match it. The source code will continue to use the old variable and class names based on “source”. Various documentation enhancements: Better explain what local sync is and how it involves two sync configs. “originating config” gets introduces instead of just “sync config”. Better explain the relationship between contexts, sync configs, and source configs (“a sync config can use the datastore configs in the same context”). An entire section on config properties in the terminology section. “item” added (Todd Wilson correctly pointed out that it was missing). Less focus on conflict resolution, as suggested by Graham Cobb. Fix examples that became invalid when fixing the password storage/lookup mechanism for GNOME keyring in 1.4. The “command line conventions”, “Synchronization beyond SyncML” and “CalDAV and CardDAV” sections were updated. It’s possible that the other sections also contain slightly incorrect usage of the terminology or are simply out-dated. * local sync: allow config name in syncURL=local:// Previously, only syncURL=local://@ was allowed and used the “target-config@context name” config as target side in the local sync. Now “local://config-name@context-name” or simply “local://config-name” are also allowed. “target-config” is still the fallback if only a context is give. It also has one more special meaning: “–configure target-config@google” will pick the “Google” template automatically because it knows that the intention is to configure the target side of a local sync. It does not know that when using some other name for the config, in which case the template (if needed) must be specified explicitly. The process name in output from the target side now also includes the configuration name if it is not the default “target-config”. * Google: remove SyncML template, combine CalDAV/CardDAV Google has turned off their SyncML server, so the corresponding “Google Contacts” template became useless and needs to be removed. It gets replaced by a “Google” template which combines the three different URLs currently used by Google for CalDAV/CardDAV. This new template can be used to configure a “target-config@google” with default calendar and address book database already enabled. The actual URL of these databases will be determined during the first sync using them. The template relies on the WebDAV backend’s new capability to search multiple different entries in the syncURL property for databases. To avoid listing each calendar twice (once for the legacy URL, once with the new one) when using basic username/password authentication, the backend needs a special case for Google and detect that the legacy URL does not need to be checked. * config: allow storing credentials for email address When configuring a WebDAV server with username = email address and no URL (which is possible if the server supports service discovery via the domain in the email address), then storing the credentials in the GNOME keyring used to fail with “cannot store password in GNOME keyring, not enough attributes”. That is because GNOME keyring seemed to get confused when a network login has no server name and some extra safeguards were added to SyncEvolution to avoid this. To store the credentials in the case above, the email address now gets split into user and domain part and together get used to look up the password. Upgrading from releases <= 1.3.99.4: ———————————— If the value of “username/databaseUser/proxyUser” contains a colon, the “user:” prefix must be added to the value, to continue treating it like a plain user name and not some reference to an unknown identity provider (like “id:”, “goa:”, “signon:”, etc.). The lookup of passwords in GNOME Keyring was updated slightly in 1.3.99.5. It may be necessary to set passwords anew if the old one is no longer found. Upgrading from release 1.2.x: —————————– The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts): syncevolution –configure \ syncFormat=text/x-vcard \ mobical addressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually: syncevolution –configure \ enableRefreshSync=TRUE \ funambol Upgrading from releases before 1.2: ———————————– Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Source, Installation, Further information ========================================= Source code bundles for users are available in http://downloads.syncevolution.org/syncevolution/sources and the original source is in [the git repositories](http://cgit.freedesktop.org/SyncEvolution/). i386, lpia and amd64 binaries for Debian-based distributions are available via the “unstable” syncevolution.org repository. Add the following entry to your /apt/source.list:
debhttp://downloads.syncevolution.org/aptunstablemain Then install “syncevolution-evolution”, “syncevolution-kde” and/or “syncevolution-activesync”. These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 10.04 LTS (Lucid), except for ActiveSync binaries which were compiled for Debian Wheezy, Ubuntu Saucy and Ubuntu Trusty. A backend for Ubuntu Online Accounts was compiled on Ubuntu Saucy. The packages mentioned above are meta-packages which pull in suitable packages matching the distro during installation. Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/). In contrast to 0.8.x archives, the 1.x .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps. More specific [HOWTOs](/wiki/howto) can be found in the Wiki.
This release enhances CalDAV/CardDAV and PBAP syncing and fixes some problems. The enhanced conflict handling introduced 1.4.99.2 was unintentionally limited to syncs with EDS on the server side; it is now also available for example in WebDAV<->SyncML bridge setups. Details: * CardDAV: implement read-ahead Instead of downloading contacts one-by-one with GET, SyncEvolution now looks at contacts that are most likely going to be needed soon and gets all of them at once with addressbook-multiget REPORT. The number of contacts per REPORT is 50 by default, configurable by setting the SYNCEVOLUTION_CARDDAV_BATCH_SIZE env variable. This has two advantages: - It avoids round-trips to the server and thus speeds up a large download (100 small contacts with individual GETs took 28s on a fast connection, 3s with two REPORTs). - It reduces the overall number of requests. Google CardDAV is known to start issuing intermittent 401 authentication errors when the number of contacts retrieved via GET is too large. Perhaps this can be avoided with addressbook-multiget. * Google Calendar: remove child hack, improve alarm hack ([FDO #63881](https://bugs.freedesktop.org/show_bug.cgi?id=63881)) Google recently enhanced support for RECURRENCE-ID, so SyncEvolution no longer needs to replace the property when uploading a single detached event with RECURRENCE-ID. However, several things are still broken in the server, with no workaround in SyncEvolution: - Removing individual events gets ignored by the server; a full “wipe out server data” might work (untested). - When updating the parent event, all child events also get updated even though they were included unchanged in the data sent by SyncEvolution. - The RECURRENCE-ID of a child event of an all-day recurring event does not get stored properly. - The update hack seems to fail for complex meetings: uploading them once and then deleting them seems to make uploading them again impossible. All of these issues were reported to Google and are worked on there, so perhaps the situation will improve. In the meantime, syncing with Google CalDAV should better be limited to: - Downloading a Google calendar in one-way mode. - Two-way syncing of simple calendars without complex meeting serieses. While updating the Google workarounds, the alarm hack (sending a new event without alarms twice to avoid the automatic server side alarm) was simplified. Now the new event gets sent only once with a pseudo-alarm. * ephemeral sync: don’t write binfile client files ([FDO #55921](https://bugs.freedesktop.org/show_bug.cgi?id=55921)) When doing PBAP caching, we don’t want any meta data written because the next sync would not use it anyway. With the latest libsynthesis we can configure “/dev/null” as datadir for the client’s binfiles and libsynthesis will avoid writing them. The PIM manager uses this for PBAP syncing automatically. For testing it can be enabled by setting the SYNCEVOLUTION_EPHEMERAL env variable. * PBAP: avoid empty field filter Empty field filter is supposed to mean “return all supported fields”. This used to work and stopped working with Android phones after an update to 4.3 (seen on Galaxy S3); now the phone only returns the mandatory TEL, FN, N fields. The workaround is to replace the empty filter list with the list of known and supported properties. This means we only pull data we really need, but it also means we won’t get to see any additional properties that the phone might support. * PBAP: transfer in chunks ([FDO #77272](https://bugs.freedesktop.org/show_bug.cgi?id=77272)) If enabled via env variables, PullAll transfers will be limited to a certain numbers contacts at different offsets until all data got pulled. See PBAP README for details. When transfering in chunks, the enumeration of contacts for the engine no longer matches the PBAP enumeration. Debug output uses “offset #x” for PBAP and “ID y” for the engine. * PBAP: remove transfer via pipe Using a pipe was never fully supported by obexd (blocks obexd). Transfering in suitably sized chunks ([FDO #77272](https://bugs.freedesktop.org/show_bug.cgi?id=77272)) will be a more obexd friendly solution with a similar effect (not having to buffer the entire address book in memory). * engine: enable batching by default ([FDO #52669](https://bugs.freedesktop.org/show_bug.cgi?id=52669)) This reverts commit c435e937cd406e904c437eec51a32a6ec6163102. Commit 7b636720a in libsynthesis fixes an unitialized memory read in the asynchronous item update code path. Testing confirms that we can now used batched writes reliably with EDS (the only backend currently supporting asynchronous writes + batching), so this change enables it again also for local and SyncEvolution<->SyncEvolution sync (with asynchronous execution of contact add/update overlapped with SyncML message exchanges) and other SyncML syncs (with changes combined into batches and executed at the end of each message). * datatypes: fix contact caching Adding grouping to the contact datatype in 1.4.99.2 broke PBAP caching: when sending an empty URL, for example, during the sync, the parsed contact had different field arrays than the locally stored contact, because the latter was saved without the empty URL. This caused the field-based comparison to detect a difference even when the final, reencoded contact wasn’t different at all. To solve this, syncing now uses the same “don’t send empty properties” configuration as local storages. Testing shows that this resolves the difference for EDS. A more resilient solution would be to add a check based on the encoded data, but that’s more costly performance wise. * datatypes: fix vCard handling The new “preserve repeating properties during conflict resolution” feature from 1.4.99.2 was only active when using EDS as storage. The relevant merge script must be applied to all datatypes, not just the EDS flavor. The feature was also unintentionally active when running in caching mode. This caused two problems: - The cached item was updated even though only the ordering of repeating properties had been modified during merging. - The merged item was sent back to the client side, which was undesirable (caching is supposed to be one-way) or even impossible (PBAP is read-only, causing sync failures eith error 20030). We must check for caching mode and disable merging when it is active. We also must not tell the engine that we updated the photo property in the winning item, because then that item would get sent to the read-only side of the sync. Perhaps a better solution would be to actually tell the engine that the remote side is read-only when we activate caching mode. * datatypes: avoid PHOTO corruption during merge ([FDO #77065](https://bugs.freedesktop.org/show_bug.cgi?id=77065)) When handling an update/update conflict (both sides of the sync have an updated contact) and photo data was moved into a local file by EDS, the engine merged the file path and the photo data together and thus corrupted the photo. The engine does not know about the special role of the photo property. This needs to be handled by the merge script, and that script did not cover this particular situation. Now the loosing side is cleared, causing the engine to then copy the winning side over into the loosing one. Found by Renato Filho/Canonical when testing SyncEvolution for Ubuntu 14.04. * PBAP syncing: updated photo not always stored Because photo data was treated like a C string, changes after any embedded null byte were ignored during a comparison. * PIM: fix phone number normalization The parsed number always has a country code, whereas SyncEvolution expected it to be zero for strings without an explicit country code. This caused a caller ID lookup of numbers like “089788899” in DE to find only telephone numbers in the current default country, instead of being more permissive and also finding “+189788899”. The corresponding unit test was broken and checked for the wrong result. Found while investigating an unrelated test failure when updating libphonenumber. * Various compiler problems and warnings fixed; compiles with –with-warnings=fatal on current Debian Testing and Ubuntu Trusty ([FDO #79316](https://bugs.freedesktop.org/show_bug.cgi?id=79316)). Upgrading from releases <= 1.3.99.4: ———————————— If the value of “username/databaseUser/proxyUser” contains a colon, the “user:” prefix must be added to the value, to continue treating it like a plain user name and not some reference to an unknown identity provider (like “id:”, “goa:”, “signon:”, etc.). The lookup of passwords in GNOME Keyring was updated slightly in 1.3.99.5. It may be necessary to set passwords anew if the old one is no longer found. Upgrading from release 1.2.x: —————————– The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts): syncevolution –configure \ syncFormat=text/x-vcard \ mobical addressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually: syncevolution –configure \ enableRefreshSync=TRUE \ funambol Upgrading from releases before 1.2: ———————————– Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Source, Installation, Further information ========================================= Source code bundles for users are available in http://downloads.syncevolution.org/syncevolution/sources and the original source is in [the git repositories](http://cgit.freedesktop.org/SyncEvolution/). i386, lpia and amd64 binaries for Debian-based distributions are available via the “unstable” syncevolution.org repository. Add the following entry to your /apt/source.list:
debhttp://downloads.syncevolution.org/aptunstablemain Then install “syncevolution-evolution”, “syncevolution-kde” and/or “syncevolution-activesync”. These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 10.04 LTS (Lucid), except for ActiveSync binaries which were compiled for Debian Wheezy, Ubuntu Saucy and Ubuntu Trusty. A backend for Ubuntu Online Accounts was compiled on Ubuntu Saucy. The packages mentioned above are meta-packages which pull in suitable packages matching the distro during installation. Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/). In contrast to 0.8.x archives, the 1.x .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps. More specific [HOWTOs](/wiki/howto) can be found in the Wiki.
Here’s a copy of the slides that I presented at the Tizen Developer Conference 2014 in San Francisco. Compared to the previous talk at the Automotive Linux Summit in Edinburgh, you’ll find Tizen-specific instructions for getting started and information about the new OAuth2 (or, as a fallback, password based) Google Contact sync. *[Update]*: a copy of the slides and an audio recording of the talk are now also on the [conference web site](https://www.tizen.org/events/tizen-developer-conference/2014/conference-schedule) (search for “PIM in IVI - Contacts, your phone, your car”).
1.4.99.2 is a new development snapshot. It enhances interoperability with CardDAV servers and in particular Google Contacts considerably. Contact data gets converted to and from the format typically used by CardDAV servers, so now anniversary, spouse, manager, assistant and instant message information are exchanged properly. Categories are not supported by Google CardDAV and thus still get lost. Custom labels get stored in EDS as extensions and no longer get lost when updating some other aspects of a contact. However, Evolution does not show custom labels and removes them when editing a property which has a custom label (BGO #730636). Scanning for CardDAV/CalDAV resources was enhanced. It now finds additional calendars with Google CalDAV and works with iCloud. However, syncing with iCloud ran into a server bug (reported as 17001498 “CalDAV REPORT drops calendar data”) and needs further work. The Ubuntu Online Accounts backend was added to syncevolution.org binaries, targeting Ubuntu Saucy and later. Details: * vcard profile: avoid data loss during merging When resolving a merge conflict, repeating properties were taken wholesale from the winning side (for example, all email addresses). If a new email address had been added on the loosing side, it got lost. Arguably it is better to preserve as much data as possible during a conflict. SyncEvolution now does that in a merge script by checking which properties in the loosing side do not exist in the winning side and copying those entries. Typically only the main value (email address, phone number) is checked and not the additional meta data (like the type). Otherwise minor differences (for example, both sides have same email address, but with different types) would lead to duplicates. Only addresses are treated differently: for them all attributes (street, country, city, etc.) are compared, because there is no single main value. * engine: UID support in contact data Before, the UID property in a vCard was ignored by the engine. Backends were responsible for ensuring that the property is set if required by the underlying storage. This turned out to be handled incompletely in the WebDAV backend. This change moves this into the engine: - UID is now field. It does not get used for matching because the engine cannot rely on it being stored by both sides. - It gets parsed if present, but only generated if explicitly enabled (because that is the traditional behavior). - It is never shown in the DevInf’s CtCap because the Synthesis engine would always show it regardless whether a rule enabled the property. That’s because rules normally only get triggered after exchanging DevInf and thus DevInf has to be rule-independent. We don’t want it shown because then merging the incoming item during a local sync would use the incoming UID, even if it is empty. - Before writing, ensure that UID is set. When updating an existing item, the Synthesis engine reads the existing item, preserves the existing UID unless the peer claims to support UID, and then updates with the existing UID. This works for local sync (where SyncEvolution never claims to support UID when talking to the other side). It will break with peers which have UID in their CtCap although they rewrite the UID and backends whose underlying storage cannot handle UID changes during an update (for example, CardDAV). * CardDAV: use Apple/Google/CardDAV vCard flavor In principle, CardDAV servers support arbitrary vCard 3.0 data. Extensions can be different and need to be preserved. However, when multiple different clients or the server’s Web UI interpret the vCards, they need to agree on the semantic of these vCard extensions. In practice, CardDAV was pushed by Apple and Apple clients are probably the most common clients of CardDAV services. When the Google Contacts Web UI creates or edits a contact, Google CardDAV will send that data using the vCard flavor used by Apple. Therefore it makes sense to exchange contacts with *all* CardDAV servers using that format. This format could be made configurable in SyncEvolution on a case-by-case basis; at the moment, it is hard-coded. During syncing, SyncEvolution takes care to translate between the vCard flavor used internally (based on Evolution) and the CardDAV vCard flavor. This mapping includes: X-AIM/JABBER/… <-> IMPP + X-SERVICE-TYPE Any IMPP property declared as X-SERVICE-TYPE=AIM will get mapped to X-AIM. Same for others. Some IMPP service types have no known X- property extension; they are stored in EDS as IMPP. X- property extensions without a known X-SERVICE-TYPE (for example, GaduGadu and Groupwise) are stored with X-SERVICE-TYPE values chosen by SyncEvolution so that Google CardDAV preserves them (GroupWise with mixed case got translated by Google into Groupwise, so the latter is used). Google always sends an X-ABLabel:Other for IMPP. This is ignored because the service type overrides it. The value itself also gets transformed during the mapping. IMPP uses an URI as value, with a chat protocol (like “aim” or “xmpp”) and some protocol specific identifier. For each X- extension the protocol is determined by the property name and the value is the protocol specific identifier without URL encoding. X-SPOUSE/MANAGER/ASSISTANT <-> X-ABRELATEDNAMES + X-ABLabel The mapping is based on the X-ABLabel property attached to the X-ABRELATEDNAMES property. This depends on the English words “Spouse”, “Manager”, “Assistant” that Google CardDAV and Apple devices seem to use regardless of the configured language. As with IMPP, only the subset of related names which have a corresponding X- property extension get mapped. The rest is stored in EDS using the X-ABRELATEDNAMES property. X-ANNIVERSARY <-> X-ABDATE Same here, with X-ABLabel:Anniversary as the special case which gets mapped. X-ABLabel parameter <-> property CardDAV vCards have labels attached to arbitrary other properties (TEL, ADR, X-ABDATE, X-ABRELATEDNAMES, …) via vCard group tags: item1.X-ABDATE:2010-01-01 item1.X-ABLabel:Anniversary The advantage is that property values can contain arbitrary characters, including line breaks and double quotation marks, which is not possible in property parameters. Neither EDS nor KDE (judging from the lack of responses on the KDE-PIM mailing list) support custom labels. SyncEvolution could have used grouping as it is done in CardDAV, but grouping is not used much (not at all?) by the UIs working with the vCards in EDS and KDE. It seemed easier to use a new X-ABLabel parameter. Characters which cannot be stored in a parameter get converted (double space to single space, line break to space, etc.) during syncing. In practice, these characters don’t appear in X-ABLabel properties anyway because neither Apple nor Google UIs allow entering them for custom labels. The “Other” label is used by Google even in case where it adds no information. For example, all XMPP properties have an associated X-ABLabel=Other although the Web UI does not provide a means to edit or show such a label. Editing the text before the value in the UI changes the X-SERVICE-TYPE parameter value, not the X-ABLabel as for other fields. Therefore the “Other” label is ignored by removing it during syncing. X-EVOLUTION-UI-SLOT (the parameter used in Evolution to determine the order of properties in the UI) gets stored in CardDAV. The only exception is Google CardDAV which got confused when an IMPP property had both X-SERVICE-TYPE and X-EVOLUTION-UI-SLOT parameters set. For Google, X-EVOLUTION-UI-SLOT is only sent on other properties and thus ordering of chat information can get lost when syncing with Google. * synccompare: support grouping and quoted parameter strings Grouped properties are sorted first according to the actual property name, then related properties are moved to the place where their group tag appears first. The first grouped property gets a “- ” prefix, all following ones are just indended with ” “. The actual group tag is not part of the normalized output, because its value is irrelevant: BDAY:19701230 - EMAIL:john@custom.com X-ABLabel:custom-label2 … FN:Mr. John 1 Doe Sr. - IMPP;X-SERVICE-TYPE=AIM:aim:aim X-ABLabel:Other … - X-ABDATE:19710101 X-ABLabel:Anniversary Redundant tags (those set for only a single property, X-ABLabel:Other) get removed as part of normalizing an item. * WebDAV: use server’s order when listing collections When doing a recursive scan of the home set, preserve the order of entries as reported by the server and check the first one first. The server knows better which entries are more relevant for the user (and thus should be the default) or may have some other relevant order. Previously, SyncEvolution replaced that order with sorting by URL, which led to a predictable, but rather meaningless order. For example, Google lists the users own calendar first, followed by the shared calendars sorted alphabetical by their name. Now SyncEvolution picks the main calendar as default correctly when scanning from https://www.google.com/calendar/dav/. * WebDAV: improved database search (Google, Zimbra) Zimbra has a principal URL that also serves as home set. When using it as start URL, SyncEvolution only looked the URL once, without listing its content, and thus did not find the databases. When following the Zimbra principal URL indirectly, SyncEvolution did check all of the collections there recursively. Unfortunately that also includes many mail folders, causing the scan to abort after checking 1000 collections (an internal safe guard). The solution for both includes tracking what to do with a URL. For the initial URL, only meta data about the URL itself gets checked. Recursive scanning is only done for the home set. If that home set contains many collections, scanning is still slow and may run into the internal safe guard limit. This cannot be avoided because the CalDAV spec explicitly states that the home set may contain normal collections which contain other collections, so a client has to do the recursive scan. When looking at a specific calendar, Google CalDAV does not report what the current principal or the home set is and therefore SyncEvolution stopped after finding just the initial calendar. Now it detects the lack of meta information and adds all parents also as candidates that need to be looked at. The downside of this is that it doesn’t know anything about which parents are relevant, so it ends up checking https://www.google.com/calendar/ and https://www.google.com/. In both cases Basic Auth gets rejected with a temporary redirect to the Google login page, which is something that SyncEvolution must ignore immediately during scanning without applying the resend workaround for “temporary rejection of valid credentials” that can happen for valid Google CalDAV URLs. * WebDAV: enhanced database search (Google Calendar) Additional databases where not found for several reasons. SyncEvolution ignored all shared calendars (http://calendarserver.org/ns/shared) and Google marks the additional calendars that way. The other problem was that the check for leaf collections (= collections which cannot contain other desired collections) incorrectly excluded those collections instead of only preventing listing of their content. With this change, https://www.google.com/calendar/dav/?SyncEvolution=Google can be used as starting point for Google Calendar. * WebDAV: fix database scan on iCloud The calendar home set URL on iCloud (the one ending in /calendars/) is declared as containing calendar data. That was enough for SyncEvolution to accept it incorrectly as calendar. However, the home set only contains calendar data indirectly. * WebDAV: support redirects between hosts and DNS SRV lookup based on URL When finding a new URL, we must be prepared to reinitialize the Neon session with the new host settings. iCloud does not have .well-known support on its www.icloud.com server. To support lookup with a non-icloudd.com email address, we must do DNS SRV lookup when access to .well-known URLs fails. We do this without a www prefix on the host first, because that is what happens to work for icloud.com. With these changes it becomes possible to do database scans on Apple iCloud, using syncURL=https://www.icloud.com or syncURL=https://icloud.com. Giving the syncURL like this is only necessary for a username that does not end in @icloud.com. When the syncURL is not set, the domain for DNS SRV lookup is taken from the username. * WebDAV: more efficient item creation PUT has the disadvantage that a client needs to choose a name and then figure out what the real name on the server is. With Google CardDAV that requires sending another request and only works because the server happens to remember the original name (which is not guaranteed!). POST works for new items without a name and happens to be implemented by Google such that the response already includes all required information (new name and revision string). POST is checked for as described in RFC 5995 once before creating a new item. Servers which don’t support it continue to get a PUT. * WebDAV: send “User-Agent: SyncEvolution” Apple iCloud servers reject requests unless they contain a User-Agent header. The exact value doesn’t seem to matter. Making the string configurable might be better, but can still be done later when it is more certain whether and for what it is needed. * WebDAV: refactor and fix DNS SRV lookup The syncevo-webdav-lookup script was not packaged. It did not report “not found” DNS results correctly and the caller did not check for this either, so when looking up the information for a domain which does not have DNS SRV entries, SyncEvolution ended up retrying for while as if there had been a temporary lookup problem. * signon: make Accounts optional The new “signon” provider only depends on lib[g]signon-glib. It uses gSSO if found, else UOA. Instead of pulling parameters and the identity via libaccounts-glib, the user of SyncEvolution now has to ensure that the identity exists and pass all relevant parameters in the “signon:” username. * gSSO: adapt to gSSO >= 2.0 * config templates: Funambol URLs Funambol turned of the URL redirect from my.funambol.com to onemedia.com. The Funambol template now uses the current URL. Users with existing Funambol configs must updated the syncURL property manually to https://onemediahub.com/sync Kudos to Daniel Clement for reporting the change. * command line: fix –update from directory The “–update
The first bug fix release in the 1.4 series addresses some issues which occurred on some systems. Several issues with Akonadi were fixed. Details: ——— * EDS: only load one backend plugin of each kind SyncEvolution was meant to load the syncecal or syncebook shared object which uses the most recent libraries (libical, libecal/libebook) on the system and then stop loooking for alternatives. Due to a string handling bug the check for already backends always found nothing, leading to multiple conflicting backends loaded on some systems (for example, those with libical0 and libical1 installed). If that happened, the backend became unusable. * ical: workaround for libical 1.0 builtin timezone change libical 1.0 started to return VTIMEZONE definitions with multiple absolute transition times instead of RRULEs. This causes problems when exchanging data with peers (see https://sourceforge.net/p/freeassociation/bugs/95/). In SyncEvolution, this affected sending an event using New Zealand time in vCalendar 1.0 format to a phone, because the internal, out-dated definition of the time zone in libsynthesis was used as fallback when loading RRULE-based timezone definitions from libical failed (see “[SyncEvolution] Some events showing wrong time on phone”). It might also affect exchanging data with CalDAV peers (not tested). The workaround is to include the original code from libical. * dbus-session.sh: create XDG_RUNTIME_DIR More recent distros (for example, Ubuntu Saucy) rely on XDG_RUNTIME_DIR. Each time dbus-session.sh runs, it must ensure that the runtime dir exists and is empty. This was a problem when trying to run activesyncd + SyncEvolution on a headless Ubuntu Saucy server (see [FDO #76273](https://bugs.freedesktop.org/show_bug.cgi?id=76273)). * Akonadi: support KDE Notes, enhanced “database” check The KDE Notes resources store items under a different MIME type than the one used in AKonadi (see “[Kde-pim] note format”). SyncEvolution use the same type as Akonadi and thus did not find existing KDE Notes resources. To support both while KDE and Akonadi transition to the same type, SyncEvolution now looks for notes resources using both MIME types and accepts both kinds of items when reading. When writing, SyncEvolution picks the MIME type that is supported by the resource, which hopefully avoids confusing the KDE app using the resource (untested). As a positive side effect, the “database” value used for opening a resource is now checked more thoroughly. Non-existent resources and the type mismatches like pointing a “kde-contacts” backend to a calendar resource are now detected early. * Akonadi: ensure that UID is set ([FDO #74342](https://bugs.freedesktop.org/show_bug.cgi?id=74342)) Akonadi resources do not enforce iCalendar 2.0 semantic like “each VEVENT must have a UID” (see “[Kde-pim] iCalendar semantic”). When receiving an event from a peer which itself does not enforce that semantic (Funambol, vCalendar 1.0 based phones), then we need to generate a UID, otherwise KOrganizer will ignore the imported event. * Akonadi: avoid threading problem in HTTP server mode ([FDO #75672](https://bugs.freedesktop.org/show_bug.cgi?id=75672)) When used as storage in a server, Akonadi got called in a background thread that gets created to handle slow initialization of sources and preventing ensuing timeouts in HTTP clients (probably not needed for Akonadi itself, but may still be useful when combining it with other sources). Akonadi cannot be used like that, leading to false “Akonadi not running” errors or (if one got past that check) failing item operations. * autotools: Add QtCore include path to KDEPIM_CFLAGS ([FDO #75670](https://bugs.freedesktop.org/show_bug.cgi?id=75670)) This fixes an issue where configure fails to find Akonadi when test programs do not compile because QString is not found. * Enhanced testing again: faster execution, less false negatives under load. Re-enabled testing of Akonadi. Upgrading from releases <= 1.3.99.4: ———————————— If the value of “username/databaseUser/proxyUser” contains a colon, the “user:” prefix must be added to the value, to continue treating it like a plain user name and not some reference to an unknown identity provider (like “id:”, “goa:”, “signon:”, etc.). The lookup of passwords in GNOME Keyring was updated slightly in 1.3.99.5. It may be necessary to set passwords anew if the old one is no longer found. Upgrading from release 1.2.x: —————————– The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts): syncevolution –configure \ syncFormat=text/x-vcard \ mobical addressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually: syncevolution –configure \ enableRefreshSync=TRUE \ funambol Upgrading from releases before 1.2: ———————————– Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Source, Installation, Further information ========================================= Source code bundles for users are available in http://downloads.syncevolution.org/syncevolution/sources and the original source is in [the git repositories](http://cgit.freedesktop.org/SyncEvolution/). i386, lpia and amd64 binaries for Debian-based distributions are available via the “stable” syncevolution.org repository. Add the following entry to your /apt/source.list:
debhttp://downloads.syncevolution.org/aptstablemain Then install “syncevolution-evolution”, “syncevolution-kde” and/or “syncevolution-activesync”. These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 10.04 LTS (Lucid), except for ActiveSync binaries which were compiled for Debian Wheezy, Ubuntu Saucy and Ubuntu Trusty. The packages mentioned above are meta-packages which pull in suitable packages matching the distro during installation. Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/). In contrast to 0.8.x archives, the 1.x .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps. More specific [HOWTOs](/wiki/howto) can be found in the Wiki.
The 1.4 release of SyncEvolution replaces 1.3.2 as the stable, supported release. 1.4 is the first stable version with the in-vehicle infotainment (IVI) PIM Manager included. GENIVI Diagnostic Log and Trace (DLT) is also supported. For more information about this aspect of SyncEvolution, see the PBAP and PIM entries in the [1.3.99 release notes](http://cgit.freedesktop.org/SyncEvolution/syncevolution/tree/NEWS?id=syncevolution-1-3-99-7) and these [Automotive Linux Summit slides](https://syncevolution.org/blogs/pohly/2013/pim-its-all-about-contacts). The biggest change for normal Linux users is Google CalDAV/CardDAV authentication with OAuth2. These are the open protocol that Google currently supports and thus the recommended way of syncing with Google, replacing ActiveSync and SyncML (both no longer available to all Google customers). Support for Google CardDAV is new. Like Evolution, SyncEvolution does not yet support some of the advanced features of the server, in particular custom labels for phone numbers, emails and addresses. Likewise, some client properties are not supported by the server: CALURI, CATEGORIES, FBURL, GEO and ROLE are not supported. Of ORG, only the first two components are supported. Currently, properties not supported by one side get lost in a full roundtrip sync. Instant Messaging information is supported by both sides with different vCard extensions; the server stores these extensions without showing the information, while SyncEvolution drops the data sent by the server. SyncEvolution depends on external components for OAuth2. It can be compiled to use [gSSO](https://01.org/gsso) or GNOME Online Accounts. The latter is enabled in binaries from syncevolution.org. GNOME Online Accounts >= 3.10 works out of the box for CalDAV and CardDAV. 3.8 is guaranteed to work for CalDAV and may also work for CardDAV, if the Linux distribution ships a patched version (like Debian Testing/Jessie does). If it does not, then GNOME Online Accounts 3.8 binary can be patched to also support CardDAV, see the GOA README. Anything older than 3.8 does not work. Support for Ubuntu Online Accounts is available when compiling from source. For setup instructions see the [GOA](http://cgit.freedesktop.org/SyncEvolution/syncevolution/plain/src/backends/goa/README) and [signon](http://cgit.freedesktop.org/SyncEvolution/syncevolution/plain/src/backends/signon/README) READMEs. Binary packages of 1.4 on syncevolution.org have enhanced support for recent distros. They now work with EDS >= 3.6 *and* < 3.6. Distros with libical1 like Ubuntu Saucy are also supported. The HTTP server became better at handling message resends when the server is slow with processing a message. The server is able to keep a sync session alive while loading the initial data set by sending acknowledgment replies before the client times out. Some issues in CalDAV, WebDAV and SyncML were fixed. Graham R. Cobb contributed several patches for enhancing ActiveSync support and making it work with Exchange 2010. Guido Günther provided some patches addressing problems when compiling SyncEvolution for Maemo. Details: ——– * D-Bus server: support DLT ([FDO #66769](https://bugs.freedesktop.org/show_bug.cgi?id=66769)) Diagnostic Log and Trace (DLT) manages a sequence of log messages, with remote controllable level of detail. SyncEvolution optionally (can be chosen at compile time and again at runtime) uses DLT instead of its own syncevolution-log.html files. See README-DLT.rst for more information. To use the feature, configure SyncEvolution with “–enable-dbus-server=–dlt –no-syslog” * D-Bus server: fix abort when mixing auto-sync and manual operations ([FDO #73562](https://bugs.freedesktop.org/show_bug.cgi?id=73562)) When enabling auto-sync for a config and then accessing or syncing the config manually via the command line tool, the server would abort at the time when the auto-sync was originally scheduled. * D-Bus server: accept WBXML with charset in incoming connections A user reported via email that the Nokia 515 sends ‘application/vnd.syncml+wbxml; charset=UTF-8’ as type of its messages this tripped up the syncevo-http-server, leading to: [ERROR] syncevo-dbus-server: /org/syncevolution/Server: message type ‘application/vnd.syncml+wbxml; charset=UTF-8’ not supported for starting a sync * D-Bus server: command line options for controlling output and startup The system log is used by default now. New command line options can be used to change this:
-d,--duration=seconds/'unlimited'Shutdownautomaticallywhenidleforthisduration(default300seconds)-v,--verbosity=levelChooseamountofoutput,0=nooutput,1=errors,2=info,3=debug;defaultis1.--dbus-verbosity=levelChooseamountofoutputviaD-Bussignals,0=nooutput,1=errors,2=info,3=debug;defaultis2.-o,--stdoutEnableprintingtostdout(resultofoperations)andstderr(errors/info/debug).-s,--no-syslogDisableprintingtosyslog.-p,--start-pimActivatethePIMManager(=unifiedaddressbook)immediately. * D-Bus: missing out parameters in D-Bus introspection XML ([FDO #57292](https://bugs.freedesktop.org/show_bug.cgi?id=57292)) The problem was in the C++ D-Bus binding. If the method that gets bound to D-Bus returns a value, that value was ignored in the signature: int foo() => no out parameter It works when the method was declared as having a retval: void foo (int &result) => integer out parameter This problem existed for both the libdbus and the GIO D-Bus bindings. In SyncEvolution it affected methods like GetVersions(). * D-Bus server: avoid progress outside of 0-100% range For example in the new TestLocalCache.testItemDelete100, the percentage value in the ProgressChanged signal become larger than 100 and then revert to 100 at the end of the sync. Seems the underlying calculation is faulty or simply inaccurate. This is not fixed. Instead the result is just clipped to the valid range. * sync: less verbose output, shorter runtime For each incoming change, one INFO line with “received x[/out of y]” was printed, immediately followed by another line with total counts “added x, updated y, removed z”. For each outgoing change, a “sent x[/out of y]” was printed. In addition, these changes were forwarded to the D-Bus server where a “percent complete” was calculated and broadcasted to clients. All of that caused a very high overhead for every single change, even if the actual logging was off. The syncevo-dbus-server was constantly consuming CPU time during a sync when it should have been mostly idle. To avoid this overhead, the updated received/sent numbers that come from the Synthesis engine are now cached and only processed when done with a SyncML message or some other event happens (whatever happens first). To keep the implementation simple, the “added x, updated y, removed z” information is ignored completely and no longer appears in the output. * command line: implement –create/remove-database Creating a database is only possible with a chosen name. The UID is chosen automatically by the storage. Only implemented in the EDS backend. * command line: execute –export and –print-items while the source is still reading Instead of reading all item IDs, then iterating over them, process each new ID as soon as it is available. With sources that support incremental reading (only the PBAP source at the moment) that provides output sooner and is a bit more memory efficient. * command line: recover from slow sync with new sync modes The error message for an unexpected slow sync still mentioned the old and obsolete “refresh-from-client/server” sync modes. Better mention “refresh-from-local/remote”. * command line: show backend error when listing databases fails The command line swallowed errors thrown by the backend while listing databases. Instead it just showed “: backend failed”. The goal was to not distract users who accidentally access a non-functional backend. But the result is that operations like –configure or –print-databases could fail without giving the user any hint about the root cause of the issue. Now the error explanation in all its gory details is included. For example, not having activesyncd running leads to: INFO] eas_contact: backend failed: fetching folder list: GDBus.Error:org.freedesktop.DBus.Error.ServiceUnknown: The name org.meego.activesyncd was not provided by any .service files And running activesyncd without the necessary gconf keys shows up as: [INFO] eas_contact: backend failed: fetching folder list: GDBus.Error:org.meego.activesyncd.Error.AccountNotFound: Failed to find account [syncevolution@lists.intel.com] * password handling: fix usage of GNOME Keyring and KWallet ([FDO #66110](https://bugs.freedesktop.org/show_bug.cgi?id=66110)) When clients like the GTK sync-ui stored a password, it was always stored as plain text in the config.ini file by the syncevo-dbus-server. The necessary code for redirecting the password storage in a keyring (GNOME or KWallet) simply wasn’t called in that case. The command line tool, even when using the D-Bus server to run the operation, had the necessary code active and thus was not affected. Now all SyncEvolution components use the same default: use safe password storage if either GNOME Keyring or KWallet were enabled during compilation, don’t use it if not. Fixing this revealed other problems, like not being able to store certain passwords that lacked the necessary lookup criteria (like syncURL and/or username). To address this, the lookup criteria where extended and a new check was added to avoid accidentally removing other passwords. As a result, it may be possible that SyncEvolution no longer finds passwords that were stored with older versions of SyncEvolution. In such a case the passwords must be set again. * GNOME: clean up keyring access and require libgnome-keyring >= 2.20 The updated error messages now always include information about the password and libgnome-keyring error texts. A workaround is used for the “Error communicating with gnome-keyring-daemon” problem that started to appear fairly frequently in the automated testing once the keyring was actually used. The problem shows up with some additional debug messages: Gkr: received an invalid, unencryptable, or non-utf8 secret Gkr: call to daemon returned an invalid response: (null).(null)() It seems that sometimes setting up a session with GNOME keyring fails such that all further communication leads to decoding problem. There is an internal method to reset the session, but it cannot be called directly. As a workaround, fake the death of the GNOME keyring daemon and thus trigger a reconnect when retrying the GNOME keyring access. This is done by sending a D-Bus message, which will also affect other clients of GNOME keyring, but hopefully without user-visible effects. * config: enhanced password handling It is possible to configure a plain username/password combination once in SyncEvolution and then use references to it in other configurations, instead of having to set (and update) the credentials in different places. This is useful in particular with WebDAV, where credentials had to be repeated several times (target config, in each database when used as part of SyncML) or when using a service which requires several configs (Google via SyncML and CalDAV). To use this, create a sync config for a normal peer or a dedicated config just for the credentials, with “username/password/syncURL” set. The “syncURL” must be set to something identifying the peer if GNOME Keyring is used for the password storage. Then set “username”, “databaseUser” and “proxyUser” properties to “id:” and all read and write access to those properties will be redirected by SyncEvolution into that other configuration. This even works in the GTK UI. For user names which contain colons, the new “user:” format must be used. Strings without colons are assumed to be normal user names, so most old configurations should continue to work. * signon: new backend using libgsignond-glib + libaccounts-glib The code works with gSSO (https://01.org/gsso) and Ubuntu Online Accounts. * GOA: get OAuth2 tokens out of GNOME Online Accounts “username = goa:…” selects an account in GOA and retrieves the OAuth2 token from that. * WebDAV: support OAuth2 If given an authentication configuration which can handle OAuth2, then OAuth2 is used instead of plain username/password authentication. * WebDAV: support Google CardDAV, break Yahoo Google CardDAV has one peculiarity: it renames new contacts during PUT without returning the new path to the client. See also http://lists.calconnect.org/pipermail/caldeveloper-l/2013-July/000524.html SyncEvolution already had a workaround for that (PROPGET on old path, extract new path from response) which happened to work. This workaround was originally added for Yahoo, which sometimes merges contacts into existing ones. In contrast to Yahoo, Google really seems to create new items. Without some server specific hacks, the client cannot tell what happened. Because Google is currently supported and Yahoo is not, let’s change the hard-coded behavior to “renamed items are new”. * WebDAV: started testing with owndrive.com = OwnCloud * WebDAV: avoid segfault during collection lookup Avoid referencing pathProps->second when the set of paths that PROPFINDs returns is empty. Apparently this can happen in combination with Calypso. * CalDAV: more workarounds for Google CalDAV + unique IDs Google became even more strict about checking REV. Tests which reused a UID after deleting the original item started to fail sometime since middle of December 2012. * CalDAV: work around Google server regression (undeclared namespace prefix in XML) Google CalDAV for a while (December 2012 till January 2013) sent invalid XML back when asked to include CardDAV properties in a PROPFIND. This got rejected in the XML parser, which prevents syncing calendar data: Neon error code 1: XML parse error at line 55: undeclared namespace prefix In the meantime Google fixed the issue in response to a bug report via email. But the workaround, only asking for the properties which are really needed, still makes sense and thus is kept. * WebDAV: auto-discovery fix With Google Contact + CardDAV the auto-discovery failed after finding the default address book, without reporting that result. * WebDAV: don’t send Basic Auth via http proactively ([FDO #57248](https://bugs.freedesktop.org/show_bug.cgi?id=57248)) Sending basic authentication headers via http is insecure. Only do it proactively when the connection is encrypted and thus protects the information or when the server explicitly asks for it. * file backend: sub-second mod time stamps Change tracking in the file backend used to be based on the modification time in seconds. When running many syncs quickly (as in testing), that can lead to changes not being detected when they happen within a second. Now the file backend also includes the sub-second part of the modification time stamp, if available. This change is relevant when upgrading SyncEvolution: most of the items will be considered “updated” once during the first sync after the upgrade (or a downgrade) because the revision strings get calculated differently. * GTK UI: fixed two crashes - running a sync with no service selected and a 64 bit pointer problem recently discovered by Tino Keitel when compiling the Debian package with -fPIE. * packaging: compatible with EDS up to and including 3.10 and both libical.so.0 and libical.so.1 The binary packages now contain different versions of syncecal.so and syncebooks.so to cover different combinations of EDS and libical. * libical: compatibiliy mode for libical.so.0 and libical.so.1 libical 1.0 broke the ABI, leading to libical.so.1. The only relevant change for SyncEvolution is the renumbering of ICAL_*_PROPERTY enum values. We can adapt to that change at runtime, which allows us to compile once with libical.so.0, then patch executables or use dynamic loading to run with the more recent libical.so.1 if we add 1 to the known constants. * packaging: fix rpm ([FDO #73347](https://bugs.freedesktop.org/show_bug.cgi?id=73347)) After installing the syncevolution.org rpm on OpenSUSE, SyncEvolution was not starting because its shared libraries were not found unless “ldconfig” was called manually. Now the package does that automatically. * packaging: fix description The syncevolution-bundle description of both rpm and deb packagesaccidentally used the same description as syncevolution-evolution. * glib: fix double-free of source tags glib 2.39.0 (aka GNOME 3.10) as found in Ubuntu Trusty introduces warnings when g_source_remove() is passed an unknown tag. SyncEvolution did this in two cases: in both, the source callback returned false and thus caused the source to be removed by the caller. In that case, the explicit g_source_remove() is redundant and must be avoided. Such a call is faulty and might accidentally remove a new source with the same tag (unlikely though, given that tags seem to get assigned incrementally). The only noticable effect were additional error messages with different numbers: [ERROR] GLib: Source ID 9 was not found when attempting to remove it * EDS: fix compile problem with boost and EDS > 3.36 This fixes the following problem, seen with Boost 1.53.0 on altlinux when compiling for EDS >= 3.6: /usr/include/boost/smart_ptr/shared_ptr.hpp: In instantiation of ‘typename boost::detail::sp_array_access::type boost::shared_ptr::operator const [with T = char*; typename boost::detail::sp_array_access::type = void; std::ptrdiff_t = long int]’: src/backends/evolution/EvolutionSyncSource.cpp:163:38: required from here /usr/include/boost/smart_ptr/shared_ptr.hpp:663:22: error: return-statement with a value, in function returning ‘void’ [-fpermissive] make[2]: *** [src/backends/evolution/src_backends_evolution_syncecal_la-EvolutionSyncSource.lo] * EDS contacts: avoid unnecessary DB writes during slow sync Traditionally, contacts were modified shortly before writing into EDS to match with Evolution expectations (must have N, only one CELL TEL, VOICE flag must be set). During a slow sync, the engine compare the modified contacts with the unmodified, incoming one. This led to mismatches and/or merge operations which end up not changing anything in the DB because the only difference would be removed again before writing. * EDS contacts: read-ahead cache Performance is improved by requesting multiple contacts at once and overlapping reading with processing. On a fast system (SSD, CPU fast enough to not be the limiting factor), testpim.py’s testSync takes 8 seconds for a “match” sync where 1000 contacts get loaded and compared against the same set of contacts. Read-ahead with only 1 contact per query speeds that up to 6.7s due to overlapping IO and processing. Read-ahead with the default 50 contacts per query takes 5.5s. It does not get much faster with larger queries. * PBAP: add support for obexd 0.47, 0.48 and Bluez 5 obexd 0.48 is almost the same as obexd 0.47, except that it dropped the SetFilter and SetFormat methods in favor of passing a Bluex 5-style filter parameter to PullAll. * PBAP: various enhancements for efficient caching of contacts * HTTP server: handle message resends If a client gave up waiting for the server’s response and resent its message while the server was still processing the message, syncing failed with “protocol error: already processing a message” raised by the syncevo-dbus-server because it wasn’t prepared to handle that situation. The right place to handle this is inside the syncevo-http-server, because it depends on the protocol (HTTP in this case) whether resending is valid or not. It handles that now by tracking the message that is currently in processing and matching it against each new message. If it matches, the new request replaces the obsolete one without sending the message again to syncevo-dbus-server. When syncevo-dbus-server replies to the old message, the reply is used to finish the newer request. * engine: prevent timeouts in HTTP server mode HTTP SyncML clients give up after a certain timeout (SyncEvolution after RetryDuration = 5 minutes by default, Nokia e51 after 15 minutes) when the server fails to respond. This can happen with SyncEvolution as server when it uses a slow storage with many items, for example via WebDAV. In the case of slow session startup, multithreading is now used to run the storage initializing in parallel to sending regular “keep-alive” SyncML replies to the client. By default, these replies are sent every 2 minutes. This can be configured with another extensions of the SyncMLVersion property: SyncMLVersion = REQUESTMAXTIME=5m Other modes do not use multithreading by default, but it can be enabled by setting REQUESTMAXTIME explicitly. It can be disabled by setting the time to zero. The new feature depends on a libsynthesis with multithreading enabled and glib >= 2.32.0, which is necessary to make SyncEvolution itself thread-safe. With an older glib, multithreading is disabled, but can be enabled as a stop-gap measure by setting REQUESTMAXTIME explicitly. * Various testing and stability enhancements. SyncEvolution had to be made thread-safe for the HTTP timeout prevention. * Nokia: always add TYPE=INTERNET to EMAIL ([FDO #61784](https://bugs.freedesktop.org/show_bug.cgi?id=61784)) Without the explicit TYPE=INTERNET, email addresses sent to a Nokia e51 were not shown by the phone and even got lost eventually (when syncing back). This commit ensures that the type is set for all emails sent to any Nokia phone, because there may be other phones which need it and phones which don’t, shouldn’t mind. This was spot-checked with a N97 mini, which works fine with and without the INTERNET type. This behavior can be disabled again for specific Nokia phones by adding a remote rule which sets the addInternetEmail session variable to FALSE again. Non-Nokia phones can enable the feature in a similar way, by setting the variable to TRUE. * SyncML: config option for broken peers Some peers have problems with meta data (CtCap, old Nokia phones) and the sync mode extensions required for advertising the restart capability (Oracle Beehive). The default in SyncEvolution is to advertise the capability, so manual configuration is necessary when working with a peer that fails in that mode. Because the problem occurs when SyncEvolution contacts the peers before it gets the device information from the peer, dynamic rules based on the peer identifiers cannot be used. Instead the local config must already disable these extra features in advance. The “SyncMLVersion” property gets extended for this. Instead of just “SyncMLVersion = 1.0” (as before) it now becomes possible to say “SyncMLVersion = 1.0, noctcap, norestart”. “noctcap” disables sending CtCap. “norestart” disables the sync mode extensions and thus doing multiple sync cycles in the same session (used between SyncEvolution instances in some cases to get client and server into sync in one session). Both keywords are case-insensitive. There’s no error checking for typos, so beware! The “SyncMLVersion” property was chosen because it was already in use for configuring SyncML compatibility aspects and adding a new property would have been harder. * ActiveSync: added support for specifying folder names Previously, the database field was interpreted as a Collection ID. This adds logic to allow the database to be interpreted as a folder path. The logic is: 1) If the database is an empty string, pass it through (this is the most common case as it is interpreted as “use the default folder for the source type”). 2) If the database matches a Collection ID, use the ID (this is the same as the previous behaviour). 3) If the database matches a folder path name, with an optional leading “/”, use the Collection ID for the matching folder. 4) Otherwise, force a FolderSync to get the latest folder changes from the server and repeat steps 2 and 3 5) If still no match, throw an error. * ActiveSync: support for listing databases Now –print-databases scans folders on the ActiveSync server and shows suitable folders for the ActiveSync backends instead of the previous, hard-coded help text. Invoking –print-databases can be used as a workaround for “SyncFolder error: Invalid synchronization key” errors. A better solution would be to do that automatically, but there was no time to implement that. See [FDO #61869](https://bugs.freedesktop.org/show_bug.cgi?id=61869) and “[SyncEvolution] Activesync server losing state” http://thread.gmane.org/gmane.comp.mobile.syncevolution/4295 * SyncML: workarounds for broken peers Some peers have problems with meta data (CtCap, old Nokia phones) and the sync mode extensions required for advertising the restart capability (Oracle Beehive). Because the problem occurs when SyncEvolution contacts the peers before it gets the device information from the peer, dynamic rules based on the peer identifiers cannot be used. Instead the local config must already disable these extra features in advance. The “SyncMLVersion” property gets extended for this. Instead of just “SyncMLVersion = 1.0” (as before) it now becomes possible to say “SyncMLVersion = 1.0, noctcap, norestart”. “noctcap” disables sending CtCap. “norestart” disables the sync mode extensions and thus doing multiple sync cycles in the same session (used between SyncEvolution instances in some cases to get client and server into sync in one session). Both keywords are case-insensitive. There’s no error checking for typos, so beware! The “SyncMLVersion” property was chosen because it was already in use for configuring SyncML compatibility aspects and adding a new property would have been harder. * engine: local cache sync mode This patch introduces support for true one-way syncing (“caching”): the local datastore is meant to be an exact copy of the data on the remote side. The assumption is that no modifications are ever made locally outside of syncing. This is different from one-way sync modes, which allows local changes and only temporarily disables sending them to the remote side. Another goal of the new mode is to avoid data writes as much as possible. This new mode only works on the server side of a sync, where the engine has enough control over the data flow. Setting “sync” to: - “local-cache-incremental” will do an incremental sync (if possible) or a slow sync (otherwise). This is usually the right mode to use, and thus has “local-cache” as alias. - “local-cache-slow” will always do a slow sync. Useful for debugging or after (accidentally) making changes on the local side. An incremental sync will ignore such changes because they are not meant to happen, aren’t checked for to improve performance and thus will leave client and server out-of-sync! Both modes are recorded in the sync report of the local side. The target side is the client and records the normal “two-way” or “slow” sync modes. With the current SyncEvolution contact field list, first, middle and last name are used to find matches for contacts. For events, tasks and memos, time, summary and description are used. * Minor memory leak fix when using GDBus GIO: GDBusMethodInfo Also depends on a glib fix, see [BGO #695376](https://bugzilla.gnome.org/show_bug.cgi?id=695376) * build fixes Avoid -lrt in make dependencies. Add missing pcre libs to syncevo-dbus-server. sqlite backend needs “#include <stdio.h>” (patch from Mario Kicherer). * autotools: fix temp file vulnerability during compilation ([CVE-2014-1639](http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2014-1639)) We must use the temporary file that was created for us securily, not a temp file named after that file. This caused a temp file vulnerability and the real temporary files were not deleted by the script. * workarounds for warnings from g++ 4.5 Upgrading from releases <= 1.3.99.4: ———————————— If the value of “username/databaseUser/proxyUser” contains a colon, the “user:” prefix must be added to the value, to continue treating it like a plain user name and not some reference to an unknown identity provider (like “id:”, “goa:”, “signon:”, etc.). The lookup of passwords in GNOME Keyring was updated slightly in 1.3.99.5. It may be necessary to set passwords anew if the old one is no longer found. Upgrading from release 1.2.x: —————————– The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts): syncevolution –configure \ syncFormat=text/x-vcard \ mobical addressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually: syncevolution –configure \ enableRefreshSync=TRUE \ funambol Upgrading from releases before 1.2: ———————————– Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. SyncEvolution 1.3.99.7 -> 1.4 ============================= Compared to the pre-release, 1.4 mostly just enhanced the testing. Compatibility with GNOME 3.10 and a glib-related issue that existed almost forever without causing obvious problems were fixed. syncevolution.org binaries now finally work with distros using libical.so.1 (for example, Ubuntu Saucy and Trusty). Details: * autotools: fix temp file vulnerability during compilation (CVE-2014-1639) We must use the temporary file that was created for us securily, not a temp file named after that file. This caused a temp file vulnerability and the real temporary files were not deleted by the script. * glib: fix double-free of source tags glib 2.39.0 (aka GNOME 3.10) as found in Ubuntu Trusty introduces warnings when g_source_remove() is passed an unknown tag. SyncEvolution did this in two cases: in both, the source callback returned false and thus caused the source to be removed by the caller. In that case, the explicit g_source_remove() is redundant and must be avoided. Such a call is faulty and might accidentally remove a new source with the same tag (unlikely though, given that tags seem to get assigned incrementally). The only noticable effect were additional error messages with different numbers: [ERROR] GLib: Source ID 9 was not found when attempting to remove it * libical: compatibiliy mode for libical.so.0 and libical.so.1 libical 1.0 broke the ABI, leading to libical.so.1. The only relevant change for SyncEvolution is the renumbering of ICAL_*_PROPERTY enum values. We can adapt to that change at runtime, which allows us to compile once with libical.so.0, then patch executables or use dynamic loading to run with the more recent libical.so.1 if we add 1 to the known constants. Source, Installation, Further information ========================================= Source code bundles for users are available in http://downloads.syncevolution.org/syncevolution/sources and the original source is [the git repositories](http://cgit.freedesktop.org/SyncEvolution/). i386, lpia and amd64 binaries for Debian-based distributions are available via the “stable” syncevolution.org repository. Add the following entry to your /apt/source.list:
debhttp://downloads.syncevolution.org/aptstablemain Then install “syncevolution-evolution”, “syncevolution-kde” and/or “syncevolution-activesync”. These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 10.04 LTS (Lucid), except for ActiveSync binaries which were compiled for Debian Wheezy, Ubuntu Saucy and Ubuntu Trusty. The packages mentioned above are meta-packages which pull in suitable packages matching the distro during installation. Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/). In contrast to 0.8.x archives, the 1.x .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps. More specific [HOWTOs](/wiki/howto) can be found in the Wiki.
This the final release candidate for 1.4. No further changes planned unless new problems are found. Details: * SSO: support Ubuntu Online Accounts When compiling from source on recent Ubuntu it becomes possible to use Ubuntu Online Accounts for authenticating against Google’s CalDAV and CardDAV servers. * D-Bus server: fix abort when mixing auto-sync and manual operations ([FDO #73562](https://bugs.freedesktop.org/show_bug.cgi?id=73562)) When enabling auto-sync for a config and then accessing or syncing the config manually via the command line tool, the server would abort at the time when the auto-sync was originally scheduled. * D-Bus server: accept WBXML with charset in incoming connections A user reported via email that the Nokia 515 sends ‘application/vnd.syncml+wbxml; charset=UTF-8’ as type of its messages. This tripped up the syncevo-http-server, leading to: [ERROR] syncevo-dbus-server: /org/syncevolution/Server: message type ‘application/vnd.syncml+wbxml; charset=UTf-8’ not supported for starting a sync We need to strip the ‘; charset=UTF-8’ suffix also when checking for WBXML. * packaging: compatible with EDS up to and including 3.10 The packages now contain three versions of syncecal.so: - one for EDS < 3.6 - one for EDS >= 3.6 < 3.10 - one for EDS >= 3.10 with the libecal-1.2 soname patched The third flavor became necessary because EDS 3.10 accidentally changed the soname. The API and ABI actually is the same. Package meta-data was fixed to reflect the extended range of compatible EDS libraries, so syncevolution-evolution can be installed again with recent EDS. * packaging: update syncevolution-kde dependencies kdebase-runtime became kde-runtime in Debian Wheezy. Accept both as prerequisite of syncevolution-kde to allow installation on newer distros without pulling in the transitional kdebase-runtime package. * packaging: fix rpm ([FDO #73347](https://bugs.freedesktop.org/show_bug.cgi?id=73347)) After installing the syncevolution.org rpm on OpenSUSE, SyncEvolution was not starting because its shared libraries were not found unless “ldconfig” was called manually. Now the package does that automatically. * packaging: fix description The syncevolution-bundle description of both rpm and deb packagesaccidentally used the same description as syncevolution-evolution. * test improvements, integration of cppcheck and clang’s scan-build Upgrading from release 1.2.x: ————————————- The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts): syncevolution –configure \ syncFormat=text/x-vcard \ mobical addressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually: syncevolution –configure \ enableRefreshSync=TRUE \ funambol Upgrading from releases before 1.2: ———————————————- Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Source, Installation, Further information ========================================= Source code bundles for users are available in http://downloads.syncevolution.org/syncevolution/sources and the original source is [the git repositories](http://cgit.freedesktop.org/SyncEvolution/). i386, lpia and amd64 binaries for Debian-based distributions are available via the “unstable” syncevolution.org repository. Add the following entry to your /apt/source.list:
debhttp://downloads.syncevolution.org/aptunstablemain Then install “syncevolution-evolution”, “syncevolution-kde” and/or “syncevolution-activesync”. These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 10.04 LTS (Lucid), except for “syncevolution-activesync” which depends on libraries in Debian Squeeze, for example EDS 3.4. Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/). In contrast to 0.8.x archives, the 1.x .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps. More specific [HOWTOs](/wiki/howto) can be found in the Wiki.
This update focuses on SyncEvolution in IVI again. It adds support for GENIVI Diagnostic Log and Trace (DLT) and enhances searching in the unified address book. The biggest change for normal Linux desktop users is enhanced support for recent distros. Binaries on syncevolution.org now work with EDS >= 3.6 *and* < 3.6. Distros with libical1 like Ubuntu Saucy are also supported. Automated testing was updated to cover these newer platforms more thoroughly. The binaries support Google CalDAV in combination with GNOME Online Accounts (GOA) >= 3.8 and and Google CardDAV with GOA >= 3.10. To use CardDAV with GOA 3.8, one has to [patch or recompile GOA](http://cgit.freedesktop.org/SyncEvolution/syncevolution/plain/src/backends/goa/README). Support for Google CalDAV/CardDAV with Ubuntu Online Accounts requires recompilation of SyncEvolution with [one additional patch](https://bugs.freedesktop.org/show_bug.cgi?id=72263). Details: * GNOME Online Accounts: fix D-Bus problem in syncevolution.org binaries Support was included in syncevolution.org binaries, but was not tested and did not actually work due to some issue accessing the D-Bus session. * libsynthesis: partial fix batching of items The batching of contact writes introduced with SyncEvolution 1.3.99.4 caused problems with non-SyncEvolution SyncML peers when syncing contacts stored in EDS >= 3.6. EDS < 3.6 was not affected. That part is fixed. However, even in SyncEvolution<->SyncEvolution syncs another crash was found. This will require more investigation. Clearly the feature is not ready yet for general sync, so for now it is disabled by default and only enabled in the simpler PBAP sync. * libsynthesis: avoid redundant (and sometimes slow) getaddrbyname() ([FDO #70771](https://bugs.freedesktop.org/show_bug.cgi?id=70771)) The network lookup of the hostname can be slow (10 second delay when not connected) and shouldn’t be necessary anyway, so disable it. * PIM: accent-insensitive and transliterated search ([FDO #56524](https://bugs.freedesktop.org/show_bug.cgi?id=56524)) Accent-insensitive search ignores accents, using the same code as in EDS. Transliterated search ignores foreign scripts by transliterating search term and contact properties to Latin first. That one is using ICU directly in the same way as EDS, but doesn’t use the EDS ETransliterator class to avoid extra string copying. This commit changes the default behavior such that searching is by default most permissive (case- and accent-insensitive, does transliteration). Flags exist to restore more restrictive matching. * PIM: relax phone number matching Previously, the current default country was used to turn phone numbers without an explicit country code into full E164 numbers, which then had to match the search term when doing a caller ID lookup. This was inconsistent with EDS, where a weaker EQUALS_NATIONAL_PHONE_NUMBER was done. The difference is that a comparison between a number with country code matches one without if the national number of the same, regardless of the current default country. This is better because it reduces the influence of the hard to guess default country on matching. Another advantage of this change is the lower memory consumption and faster comparison, because strings are now stored in 4 + 8 byte numbers instead of strings of varying length. * PIM: fix incorrect write into pim-manager.ini ([FDO #70772](https://bugs.freedesktop.org/show_bug.cgi?id=70772)) Removing a peer accidentally wrote the updated list of active address books into the “sort” property of pim-manager.ini, which then prevented starting the PIM Manager. * PIM: ignore broken sort order in config ([FDO #70772](https://bugs.freedesktop.org/show_bug.cgi?id=70772)) Failure to set the sort order from pim-manager.ini should not prevent the startup of the PIM Manager because the client cannot really diagnose and fix the problem. It is better to try again with the default sort order. * PIM: adapt to locale changes at runtime ([FDO #66618](https://bugs.freedesktop.org/show_bug.cgi?id=66618)) Listen to signals from localed D-Bus system service and update all internal state which depends on the current locale. This state includes: - pre-computed data in all loaded contacts - filtering (for example, case sensitivity is locale dependent) - the sort order This feature can be controlled by setting the SYNCEVOLUTION_LOCALED env variable: - “session” - use a localed instance on the D-Bus session bus instead of the system instance. This was originally meant for testing, but might also be useful for per-user setting changes. - “none” - disables the feature * PIM: fix sync.py + multiple peers Due to overwriting a variable, configuring multiple different peers did not work. * D-Bus server: support DLT ([FDO #66769](https://bugs.freedesktop.org/show_bug.cgi?id=66769)) Diagnostic Log and Trace (DLT) manages a sequence of log messages, with remote controllable level of detail. SyncEvolution optionally (can be chosen at compile time and again at runtime) uses DLT instead of its own syncevolution-log.html files. See README-DLT.rst for more information. To use the feature, configure SyncEvolution with “–enable-dbus-server=–dlt –no-syslog” * EDS: enhanced compatibility mode SyncEvolution compiled for EDS < 3.6 can now also load EDS backends compiled for EDS >= 3.6. The packaging for syncevolution.org uses that to bundle EDS backends compiled on different distros in the same package. * EDS: SYNCEVOLUTION_EBOOK_QUERY env variable Setting the SYNCEVOLUTION_EBOOK_QUERY env variable to a valid EBook query string limits the results to contacts matching that query. Useful only in combination with –print-items or –export. Only implemented for EDS >= 3.6. * EDS: fix compile problem with boost and EDS > 3.36 This fixes the following problem, seen with Boost 1.53.0 on altlinux when compiling for EDS >= 3.6: /usr/include/boost/smart_ptr/shared_ptr.hpp: In instantiation of ‘typename boost::detail::sp_array_access::type boost::shared_ptr::operator const [with T = char*; typename boost::detail::sp_array_access::type = void; std::ptrdiff_t = long int]’: src/backends/evolution/EvolutionSyncSource.cpp:163:38: required from here /usr/include/boost/smart_ptr/shared_ptr.hpp:663:22: error: return-statement with a value, in function returning ‘void’ [-fpermissive] make[2]: *** [src/backends/evolution/src_backends_evolution_syncecal_la-EvolutionSyncSource.lo] * PBAP: add support for obexd 0.48 obexd 0.48 is almost the same as obexd 0.47, except that it dropped the SetFilter and SetFormat methods in favor of passing a Bluex 5-style filter parameter to PullAll. SyncEvolution now supports 4, in words, four different obexd APIs. Sigh. This feature was originally announced for SyncEvolution 1.3.99.5, but not actually included in the code yet. Upgrading from release 1.2.x: ————————————- The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts): syncevolution –configure \ syncFormat=text/x-vcard \ mobical addressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually: syncevolution –configure \ enableRefreshSync=TRUE \ funambol Upgrading from releases before 1.2: ———————————————- Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Source, Installation, Further information ========================================= Source code bundles for users are available in http://downloads.syncevolution.org/syncevolution/sources and the original source is [the git repositories](http://cgit.freedesktop.org/SyncEvolution/). i386, lpia and amd64 binaries for Debian-based distributions are available via the “unstable” syncevolution.org repository. Add the following entry to your /apt/source.list:
debhttp://downloads.syncevolution.org/aptunstablemain Then install “syncevolution-evolution”, “syncevolution-kde” and/or “syncevolution-activesync”. These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 10.04 LTS (Lucid), except for “syncevolution-activesync” which depends on libraries in Debian Squeeze, for example EDS 3.4. Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/). In contrast to 0.8.x archives, the 1.x .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps. More specific [HOWTOs](/wiki/howto) can be found in the Wiki.
I just presented the work done on PIM in Tizen IVI 3.0 at the Linux Foundation’s Automotive Linux Summit. The work is based on a considerably enhanced Evolution Data Server, libphonenumber, folks, and of course SyncEvolution. If you want to know more about this, the slides are attached.
SyncEvolution now supports Google CalDAV/CardDAV with OAuth2 authentication. These are the open protocol that Google currently supports and thus the recommended way of syncing with Google, replacing ActiveSync and SyncML (both no longer available to all Google customers). Support for Google CardDAV is new. Because of a vCard encoding issue on the server side, spaces in long notes may get removed. Like Evolution, SyncEvolution does not yet support some of the advanced features of the server, in particular custom labels for phone numbers, emails and addresses. Likewise, some client properties are not supported by the server: CALURI, CATEGORIES, FBURL, GEO and ROLE are not supported. Of ORG, only the first two components are supported. Currently, properties not supported by one side get lost in a full roundtrip sync. SyncEvolution depends on external components for OAuth2. It can be compiled to use [gSSO](http://cgit.freedesktop.org/SyncEvolution/syncevolution/plain/src/backends/signon/README) or [GNOME Online Accounts](http://cgit.freedesktop.org/SyncEvolution/syncevolution/plain/src/backends/goa/README). GNOME Online Accounts >= 3.10 works out of the box for CalDAV and CardDAV, 3.8 only for CardDAV (but the GNOME Online Accounts binary can be patched to also support CalDAV), anything older than 3.8 does not work. Support for Ubuntu Online Accounts should not be hard to add, but is [not available yet](http://thread.gmane.org/gmane.comp.mobile.syncevolution/4353/focus=4490). Evolution >= 3.6 is not supported by the binaries on syncevolution.org. On systems with a more recent Evolution, SyncEvolution must be compiled from source. Details: * GTK UI: fixed two crashes - running a sync with no service selected and a 64 bit pointer problem recently discovered by Tino Keitel when compiling the Debian package with -fPIE. * password handling: fix usage of GNOME Keyring and KWallet ([FDO #66110](https://bugs.freedesktop.org/show_bug.cgi?id=66110)) When clients like the GTK sync-ui stored a password, it was always stored as plain text in the config.ini file by the syncevo-dbus-server. The necessary code for redirecting the password storage in a keyring (GNOME or KWallet) simply wasn’t called in that case. The command line tool, even when using the D-Bus server to run the operation, had the necessary code active and thus was not affected. Now all SyncEvolution components use the same default: use safe password storage if either GNOME Keyring or KWallet were enabled during compilation, don’t use it if not. Fixing this revealed other problems, like not being able to store certain passwords that lacked the necessary lookup criteria (like syncURL and/or username). To address this, the lookup criteria where extended and a new check was added to avoid accidentally removing other passwords. As a result, it may be possible that SyncEvolution no longer finds passwords that were stored with older versions of SyncEvolution. In such a case the passwords must be set again. * GNOME: clean up keyring access and require libgnome-keyring >= 2.20 The updated error messages now always include information about the password and libgnome-keyring error texts. A workaround is used for the “Error communicating with gnome-keyring-daemon” problem that started to appear fairly frequently in the automated testing once the keyring was actually used. The problem shows up with some additional debug messages: Gkr: received an invalid, unencryptable, or non-utf8 secret Gkr: call to daemon returned an invalid response: (null).(null)() It seems that sometimes setting up a session with GNOME keyring fails such that all further communication leads to decoding problem. There is an internal method to reset the session, but it cannot be called directly. As a workaround, fake the death of the GNOME keyring daemon and thus trigger a reconnect when retrying the GNOME keyring access. This is done by sending a D-Bus message, which will also affect other clients of GNOME keyring, but hopefully without user-visible effects. * config: enhanced password handling It is possible to configure a plain username/password combination once in SyncEvolution and then use references to it in other configurations, instead of having to set (and update) the credentials in different places. This is useful in particular with WebDAV, where credentials had to be repeated several times (target config, in each database when used as part of SyncML) or when using a service which requires several configs (Google via SyncML and CalDAV). To use this, create a sync config for a normal peer or a dedicated config just for the credentials, with “username/password/syncURL” set. The “syncURL” must be set to something identifying the peer if GNOME Keyring is used for the password storage. Then set “username”, “databaseUser” and “proxyUser” properties to “id:” and all read and write access to those properties will be redirected by SyncEvolution into that other configuration. This even works in the GTK UI. For user names which contain colons, the new “user:” format must be used. Strings without colons are assumed to be normal user names, so most old configurations should continue to work. * signon: new backend using libgsignond-glib + libaccounts-glib The code works with gSSO (https://01.org/gsso). With some tweaks to the configure check and some ifdefs it probably could be made to work with Ubuntu Online Accounts. The code depends on an account accessible via libaccounts-glib which has a provider and and (optionally) services enabled for that provider. It is not necessary that the account already has a signon identity ID, the backend will create that for the provider (and thus shared between all services) if necessary. Therefore it is possible to use the ag-tool to create and enable the account and services. Provider and service templates are in the next commit. * WebDAV: support OAuth2 If given an authentication configuration which can handle OAuth2, then OAuth2 is used instead of plain username/password authentication. * WebDAV: support Google CardDAV, break Yahoo Google CardDAV has one peculiarity: it renames new contacts during PUT without returning the new path to the client. See also http://lists.calconnect.org/pipermail/caldeveloper-l/2013-July/000524.html SyncEvolution already had a workaround for that (PROPGET on old path, extract new path from response) which happened to work. This workaround was originally added for Yahoo, which sometimes merges contacts into existing ones. In contrast to Yahoo, Google really seems to create new items. Without some server specific hacks, the client cannot tell what happened. Because Google is currently supported and Yahoo is not, let’s change the hard-coded behavior to “renamed items are new”. * WebDAV: started testing with owndrive.com = OwnCloud * GOA: get OAuth2 tokens out of GNOME Online Accounts “username = goa:…” selects an account in GOA and retrieves the OAuth2 token from that. The implementation uses the GOA D-Bus API directly, because our C++ D-Bus bindings are easier to use and this avoids an additional library dependency. * PIM: fix UID usage in sync.py example Using the underscore in the UID has been wrong all along, it only happened to work because UID sanity checking was missing. After adding it, the example broke. Now simply remove the colon. It makes the UID less readable, but it doesn’t have to be, and ensures that file names and database names contain the UID as-is. * PIM: if busy, don’t shut down While there are sessions pending or active, the server should not shut down. It did that while executing a long-running PIM Manager SyncPeer() operations, by default after 10 minutes. This was not a problem elsewhere because other operations are associated with a client, whose presence also prevents shutdowns. Perhaps PIM Manager should also track the caller and treat it like a client. * PBAP: do not end Bluez5 transfer prematurely A transfer was marked as finished prematurely when encountering the “active” Status value, which can happen for longer transfers. * PBAP: add support for obexd 0.48 obexd 0.48 is almost the same as obexd 0.47, except that it dropped the SetFilter and SetFormat methods in favor of passing a Bluex 5-style filter parameter to PullAll. SyncEvolution now supports 4, in words, four different obexd APIs. Sigh. * updated tests Upgrading from releases <= 1.3.99.4: ———————————————– If the value of “username/databaseUser/proxyUser” contains a colon, the “user:” prefix must be added to the value, to continue treating it like a plain user name and not some reference to an unknown identity provider (like “id:”, “goa:”, “signon:”, etc.). The lookup of passwords in GNOME Keyring was updated slightly in 1.3.99.5. It may be necessary to set passwords anew if the old one is no longer found. Upgrading from release 1.2.x: ————————————- The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts): syncevolution –configure \ syncFormat=text/x-vcard \ mobical addressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually: syncevolution –configure \ enableRefreshSync=TRUE \ funambol Upgrading from releases before 1.2: ———————————————- Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Source, Installation, Further information ========================================= Source code bundles for users are available in http://downloads.syncevolution.org/syncevolution/sources and the original source is [the git repositories](http://cgit.freedesktop.org/SyncEvolution/). i386, lpia and amd64 binaries for Debian-based distributions are available via the “unstable” syncevolution.org repository. Add the following entry to your /apt/source.list:
debhttp://downloads.syncevolution.org/aptunstablemain Then install “syncevolution-evolution”, “syncevolution-kde” and/or “syncevolution-activesync”. These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 10.04 LTS (Lucid), except for “syncevolution-activesync” which depends on libraries in Debian Squeeze, for example EDS 3.4. Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/). In contrast to 0.8.x archives, the 1.x .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps. More specific [HOWTOs](/wiki/howto) can be found in the Wiki.
The focus of this development snapshot is enhanced performance of syncing. With EDS, contacts get added, updated or loaded with batch operations, which led to 4x runtime improvements when importing PBAP address book for the first time. Removing unnecessary work from any following PBAP sync resulted in a 6x improvement. These improvements also benefit non-PBAP syncing and could in theory work with any SyncML peer. In practice, batching of items is currently limited to SyncEvolution as peer. The PBAP backend itself was rewritten such that data gets transferred from a phone in parallel to processing the already transferred data. The effect is that on a sufficiently fast system, a sync takes about the same time as downloading all contacts. To get the text-only part of the contacts even faster, PBAP syncing can be done such that it first syncs the text-only parts (without removing existing photos), then in a second round adds or modifies photos. The PIM Manager uses this incremental mode by default, in the command line it can be chose with the SYNCEVOLUTION_PBAP_SYNC env variable. The HTTP server became better at handling message resends when the server is slow with processing a message. The server is able to keep a sync session alive while loading the initial data set by sending acknowledgement replies before the client times out. Guido Günther provided some patches addressing problems when compiling SyncEvolution for Maemo. Details: * sync: less verbose output, shorter runtime For each incoming change, one INFO line with “received x[/out of y]” was printed, immediately followed by another line with total counts “added x, updated y, removed z”. For each outgoing change, a “sent x[/out of y]” was printed. In addition, these changes were forwarded to the D-Bus server where a “percent complete” was calculated and broadcasted to clients. All of that caused a very high overhead for every single change, even if the actual logging was off. The syncevo-dbus-server was constantly consuming CPU time during a sync when it should have been mostly idle. To avoid this overhead, the updated received/sent numbers that come from the Synthesis engine are now cached and only processed when done with a SyncML message or some other event happens (whatever happens first). To keep the implementation simple, the “added x, updated y, removed z” information is ignored completely and no longer appears in the output. * HTTP server: handle message resends If a client gave up waiting for the server’s response and resent its message while the server was still processing the message, syncing failed with “protocol error: already processing a message” raised by the syncevo-dbus-server because it wasn’t prepared to handle that situation. The right place to handle this is inside the syncevo-http-server, because it depends on the protocol (HTTP in this case) whether resending is valid or not. It handles that now by tracking the message that is currently in processing and matching it against each new message. If it matches, the new request replaces the obsolete one without sending the message again to syncevo-dbus-server. When syncevo-dbus-server replies to the old message, the reply is used to finish the newer request. * PBAP: incremental sync ((FDO #59551)[https://bugs.freedesktop.org/show_bug.cgi?id=59551]) Depending on the SYNCEVOLUTION_PBAP_SYNC env variable, syncing reads all properties as configured (“all”), excludes photos (“text”) or first text, then all (“incremental”). When excluding photos, only known properties get requested. This avoids issues with phones which reject the request when enabling properties via the bit flags. This also helps with “databaseFormat=^PHOTO”. * PIM: use incremental sync for PBAP by default ((FDO #59551)[https://bugs.freedesktop.org/show_bug.cgi?id=59551]) When doing a PBAP sync, PIM manager asks the D-Bus sync helper to set its SYNCEVOLUTION_PBAP_SYNC to “incremental”. If the env variable is already set, it does not get overwritten, which allows overriding this default. * PIM: set debug level in peer configs via env variable Typically the peer configs get created from scratch, in particular when testing with testpim.py. In that case the log level cannot be set in advance and doing it via the D-Bus API is also not supported. Therefore, for debugging, use SYNCEVOLUTION_LOGLEVEL= to create peers with a specific log level. * PIM: include pim-manager-api.txt in source distro ((FDO #62516)[https://bugs.freedesktop.org/show_bug.cgi?id=62516]) The text file must be listed explicitly to be included by “make dist”. * PIM: “full name” -> “fullname” fix in documentation ((FDO #62515)[https://bugs.freedesktop.org/show_bug.cgi?id=62515]) Make the documentation match the code. A single word without space makes more sense, so let’s go with what the code already used. * PIM: enhanced searching (search part of (FDO #64177)[https://bugs.freedesktop.org/show_bug.cgi?id=64177]) Search terms now also include ‘is/contains/begins-with/ends-with’ and they can be combined with ‘and’ and ‘or’, also recursively. * PIM: Pinyin sorting for zh languages (part of (FDO #64173)[https://bugs.freedesktop.org/show_bug.cgi?id=64173]) Full interleaving of Pinyin transliterations of Chinese names with Western names can be done by doing an explicit Pinyin transliteration as part of computing the sort keys. This is done using ICU’s Transliteration(“Han-Latin”), which we have to call directly because boost::locale does not expose that API. We hard-code this behavior for all “zh” languages (as identified by boost::locale), because by default, ICU would sort Pinyin separately from Western names when using the “pinyin” collation. * PIM: new return value for SyncPeer(), new SyncProgress signal ((FDO #63417)[https://bugs.freedesktop.org/show_bug.cgi?id=63417]) The SyncPeer() result is derived from the sync statistics. To have them available, the “sync done” signal must include the SyncReport. Start and end of a sync could already be detected; “modified” signals while a sync runs depends on a new signal inside the SyncContext when switching from one cycle to the next and at the end of the last one. * PIM: allow removal of data together with database removal (part of (FDO #64835)[https://bugs.freedesktop.org/show_bug.cgi?id=64835]) There is a difference in EDS between removing the database definition from the ESourceRegistry (which makes the data unaccessible via EDS) and removing the actual database. EDS itself only removes the definition and leaves the data around to be garbage-collected eventually. This is not what we want for the PIM Manager API; the API makes a stronger guarantee that data is really gone. Fixed by introducing a new mode flag for the deleteDatabase() method and deleting the directory of the source directly in the EDS backend, if requested by the caller. The syncevolution command line tool will use the default mode and thus keep the data around, while the PIM Manager forces the removal of data. * EDS: create new databases by cloning the builtin ones ((FDO #64176)[https://bugs.freedesktop.org/show_bug.cgi?id=64176]) Instead of hard-coding a specific “Backend Summary Setup” in SyncEvolution, copy the config of the system database. That way special flags (like the desired “Backend Summary Setup” for local address books) can be set on a system-wide basis and without having to modify or configure SyncEvolution. Because EDS has no APIs to clone an ESource or turn a .source file into a new ESource, SyncEvolution has to resort to manipulating and creating the keyfile directly. * EDS contacts: update PHOTO+GEO during slow sync, avoid rewriting PHOTO file If PHOTO and/or GEO were the only modified properties during a slow sync, the updated item was not written into local storage because they were marked as compare=”never” = “not relevant”. For PHOTO this was intentional in the sample config, with the rationale that local storages often don’t store the data exactly as requested. When that happens, comparing the data would lead to unnecessary writes. But EDS and probably all other local SyncEvolution storages (KDE, file) store the photo exactly as requested, so not considering changes had the undesirable effect of not always writing new photo data. For GEO, ignoring it was accidental. * EDS contacts: avoid unnecessary DB writes during slow sync Traditionally, contacts were modified shortly before writing into EDS to match with Evolution expectations (must have N, only one CELL TEL, VOICE flag must be set). During a slow sync, the engine compare the modified contacts with the unmodified, incoming one. This led to mismatches and/or merge operations which end up not changing anything in the DB because the only difference would be removed again before writing. * EDS contacts: read-ahead cache Performance is improved by requesting multiple contacts at once and overlapping reading with processing. On a fast system (SSD, CPU fast enough to not be the limiting factor), testpim.py’s testSync takes 8 seconds for a “match” sync where 1000 contacts get loaded and compared against the same set of contacts. Read-ahead with only 1 contact per query speeds that up to 6.7s due to overlapping IO and processing. Read-ahead with the default 50 contacts per query takes 5.5s. It does not get much faster with larger queries. * command line: execute –export and –print-items while the source is still reading Instead of reading all item IDs, then iterating over them, process each new ID as soon as it is available. With sources that support incremental reading (only the PBAP source at the moment) that provides output sooner and is a bit more memory efficient. * WebDAV: avoid segfault during collection lookup Avoid referencing pathProps->second when the set of paths that PROPFINDs returns is empty. Apparently this can happen in combination with Calypso. * engine: prevent timeouts in HTTP server mode HTTP SyncML clients give up after a certain timeout (SyncEvolution after RetryDuration = 5 minutes by default, Nokia e51 after 15 minutes) when the server fails to respond. This can happen with SyncEvolution as server when it uses a slow storage with many items, for example via WebDAV. In the case of slow session startup, multithreading is now used to run the storage initializing in parallel to sending regular “keep-alive” SyncML replies to the client. By default, these replies are sent every 2 minutes. This can be configured with another extensions of the SyncMLVersion property: SyncMLVersion = REQUESTMAXTIME=5m Other modes do not use multithreading by default, but it can be enabled by setting REQUESTMAXTIME explicitly. It can be disabled by setting the time to zero. The new feature depends on a libsynthesis with multithreading enabled and glib >= 2.32.0, which is necessary to make SyncEvolution itself thread-safe. With an older glib, multithreading is disabled, but can be enabled as a stop-gap measure by setting REQUESTMAXTIME explicitly. * Various testing and stability enhancements. SyncEvolution had to be made thread-safe for the HTTP timeout prevention. Source, Installation, Further information ========================================= Source code bundles for users are available in http://downloads.syncevolution.org/syncevolution/sources and the original source is [the git repositories](http://cgit.freedesktop.org/SyncEvolution/). i386, lpia and amd64 binaries for Debian-based distributions are available via the “unstable” syncevolution.org repository. Add the following entry to your /apt/source.list:
debhttp://downloads.syncevolution.org/aptunstablemain Then install “syncevolution-evolution”, “syncevolution-kde” and/or “syncevolution-activesync”. These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 10.04 LTS (Lucid), except for “syncevolution-activesync” which depends on libraries in Debian Squeeze, for example EDS 3.4. Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/). In contrast to 0.8.x archives, the 1.x .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps. More specific [HOWTOs](/wiki/howto) can be found in the Wiki.
So far the development cycle for SyncEvolution 1.4 mostly focused on implementing the [“PIM Manager”](http://comments.gmane.org/gmane.comp.mobile.syncevolution/4009) D-Bus API for IVI use cases [1]. The 1.3.99.3 pre-release starts to include more features and bug fixes again for syncing. For example, several ActiveSync improvements from Graham Cobb were included. The remaining goal for 1.4, besides more testing of course, is to work out how to support Google CalDAV and CardDAV. I am in discussion with Google to get SyncEvolution [whitelisted](http://googleblog.blogspot.de/2013/03/a-second-spring-of-cleaning.html) for use with these APIs - fingers crossed… Upgrading from release 1.2.x —————————- The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts): syncevolution –configure \ syncFormat=text/x-vcard \ mobical addressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually: syncevolution –configure \ enableRefreshSync=TRUE \ funambol Upgrading from releases before 1.2 ———————————- Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Changes 1.3.2 -> 1.3.99.3 ========================= * PIM Manager: add ReplaceSearch, always allow it The new ReplaceSearch is more flexible than RefineSearch. It can handle both tightening the search and relaxing it. The downside of it is the more expensive implementation (must check all contacts again, then find minimal set of change signals to update view). Previously, a search which had no filter set at all at the begining could not be refined. This limitation of the implementation gets removed by always using a FilteredView, even if the initial filter is empty. * PIM Manager: introduce CreateConfig() That SetPeer() allows modifying and creating a config leads to race conditions when multiple clients want to create a config. The new CreateConfig() avoids that by atomically checking that a config does not exist yet and creating it. SetPeer() is still available for backwards compatibility. It continues to be used for modifying an existing config in TestContacts.testSync to check the effect of the logging settings. * PIM Manager: fix double entries in filtered search with limit Stressing the FilteredView by using it in tests originally written for the FullView showed that the filling up a view may have used data while it was inconsistent internally, leading to contacts being present multiple times. * PIM Manager and sync: support location = GEO property ([FDO #60373](https://bugs.freedesktop.org/show_bug.cgi?id=60373)) Exposed as “location” -> (lat, long) in the D-Bus bindings. Reading, writing and updating are supported. * PIM Manager: support groups = CATEGORIES ([FDO #60380](https://bugs.freedesktop.org/show_bug.cgi?id=60380)) Allow reading and writing of groups (folks terminology), aka CATEGORIES in vCard. * PIM Manager: intelligent phone search in EDS (part of [FDO #59571](https://bugs.freedesktop.org/show_bug.cgi?id=59571)) If phone number search is enabled in EDS, then the direct search in EDS now uses the more accurate E_BOOK_QUERY_EQUALS_NATIONAL_PHONE_NUMBER comparison, with the E164 formatted caller ID as value to compare against. This gives semantically correct results. The previous solution (now the fallback) had to use substring searches, which did not match if the contact’s phone number was not formatted according to E164 and which may have matched the wrong contacts if the trailing numbers are the same. * PIM Manager : use pre-computed normalized phone numbers from EDS (part of [FDO #59571](https://bugs.freedesktop.org/show_bug.cgi?id=59571)) When available, the pre-computed E164 number from EDS will be used instead of doing one libphonebook parser run for each telephone number while reading. Benchmarking showed that this parsing was the number one hotspot, so this is a considerable improvement. * PIM Manager: fix error messages Ensure and check that no unnecessary ERROR messages are printed. libfolks was used slightly incorrectly, leading to several harmless error messages (glib asserts). libphonenumber printed its error messages to stdout. * PIM Manager: fix memory leaks during writing of contacts Constructing the GValues created additional references instead of taking over ownership as intended. * D-Bus server: fix read-after-free bug when using syslog openlog() expects the string to remain valid. Must ensure that in LoggerSyslog by making a copy. Found with valgrind. * PIM Manager: make implementation of some of the D-Bus methods thread-safe The goal is to make it easier to extend syncevo-dbus-server with other IPC mechanisms, which then can call the native C++ code directly. That code was not prepared to handle calls in threads other than the main one. Now this is checked when entering the methods and work is shifted to the main thread if necessary. In the meantime the calling thread waits for completion. * PIM Manager: check responsiveness (part of [FDO #60851](https://bugs.freedesktop.org/show_bug.cgi?id=60851)) Enhanced the testActive test so that it can detect when the D-Bus server stops responding for too long. One major reason for that was event processing in folks, which got improved as part of https://bugzilla.gnome.org/show_bug.cgi?id=694385 * PIM Manager: adapt to gee 0.8 Changed the code to compile with gee 0.8, as used by folks 0.9.x. Older versions of folks are no longer supported. * PBAP: support Bluez 5 The new Bluez 5 API is the third supported API for doing PBAP transfers. It gets checked first, then the PBAB backend falls back to new-style obexd (file based, similar to Bluez 5, but not quite the same) and finally old-style obexd (data transfer via D-Bus). In contrast to previous APIs, Bluez 5 does not report the reason for a failed PBAP transfer. SyncEvolution then throws a generic “transfer failed” error with “reason unknown” as message. * command line: recover from slow sync with new sync modes The error message for an unexpected slow sync still mentioned the old and obsolete “refresh-from-client/server” sync modes. Better mention “refresh-from-local/remote”. * CalDAV: more workarounds for Google CalDAV + unique IDs Google became even more strict about checking REV. Tests which reused a UID after deleting the original item started to fail sometime since middle of December 2012. * CalDAV: work around Google server regression (undeclared namespace prefix in XML) Google CalDAV for a while (December 2012 till January 2013) sent invalid XML back when asked to include CardDAV properties in a PROPFIND. This got rejected in the XML parser, which prevents syncing calendar data: Neon error code 1: XML parse error at line 55: undeclared namespace prefix In the meantime Google fixed the issue in response to a bug report via email. But the workaround, only asking for the properties which are really needed, still makes sense and thus is kept. * WebDAV: don’t send Basic Auth via http proactively ([FDO #57248](https://bugs.freedesktop.org/show_bug.cgi?id=57248)) Sending basic authentication headers via http is insecure. Only do it proactively when the connection is encrypted and thus protects the information or when the server explicitly asks for it. * Nokia: always add TYPE=INTERNET to EMAIL ([FDO #61784](https://bugs.freedesktop.org/show_bug.cgi?id=61784)) Without the explicit TYPE=INTERNET, email addresses sent to a Nokia e51 were not shown by the phone and even got lost eventually (when syncing back). This commit ensures that the type is set for all emails sent to any Nokia phone, because there may be other phones which need it and phones which don’t, shouldn’t mind. This was spot-checked with a N97 mini, which works fine with and without the INTERNET type. This behavior can be disabled again for specific Nokia phones by adding a remote rule which sets the addInternetEmail session variable to FALSE again. Non-Nokia phones can enable the feature in a similar way, by setting the variable to TRUE. * SyncML: config option for broken peers Some peers have problems with meta data (CtCap, old Nokia phones) and the sync mode extensions required for advertising the restart capability (Oracle Beehive). The default in SyncEvolution is to advertise the capability, so manual configuration is necessary when working with a peer that fails in that mode. Because the problem occurs when SyncEvolution contacts the peers before it gets the device information from the peer, dynamic rules based on the peer identifiers cannot be used. Instead the local config must already disable these extra features in advance. The “SyncMLVersion” property gets extended for this. Instead of just “SyncMLVersion = 1.0” (as before) it now becomes possible to say “SyncMLVersion = 1.0, noctcap, norestart”. “noctcap” disables sending CtCap. “norestart” disables the sync mode extensions and thus doing multiple sync cycles in the same session (used between SyncEvolution instances in some cases to get client and server into sync in one session). Both keywords are case-insensitive. There’s no error checking for typos, so beware! The “SyncMLVersion” property was chosen because it was already in use for configuring SyncML compatibility aspects and adding a new property would have been harder. * ActiveSync: added support for specifying folder names Previously, the database field was interpreted as a Collection ID. This adds logic to allow the database to be interpreted as a folder path. The logic is: 1) If the database is an empty string, pass it through (this is the most common case as it is interpreted as “use the default folder for the source type”). 2) If the database matches a Collection ID, use the ID (this is the same as the previous behaviour). 3) If the database matches a folder path name, with an optional leading “/”, use the Collection ID for the matching folder. 4) Otherwise, force a FolderSync to get the latest folder changes from the server and repeat steps 2 and 3 5) If still no match, throw an error. * ActiveSync: support for listing databases Now –print-databases scans folders on the ActiveSync server and shows suitable folders for the ActiveSync backends instead of the previous, hard-coded help text. Invoking –print-databases can be used as a workaround for “SyncFolder error: Invalid synchronization key” errors. A better solution would be to do that automatically, but there was no time to implement that. See [FDO #61869](https://bugs.freedesktop.org/show_bug.cgi?id=61869) and “[SyncEvolution] Activesync server losing state” http://thread.gmane.org/gmane.comp.mobile.syncevolution/4295 * command line: show backend error when listing databases fails The command line swallowed errors thrown by the backend while listing databases. Instead it just showed “: backend failed”. The goal was to not distract users who accidentally access a non-functional backend. But the result is that operations like –configure or –print-databases could fail without giving the user any hint about the root cause of the issue. Now the error explanation in all its gory details is included. For example, not having activesyncd running leads to: INFO] eas_contact: backend failed: fetching folder list: GDBus.Error:org.freedesktop.DBus.Error.ServiceUnknown: The name org.meego.activesyncd was not provided by any .service files And running activesyncd without the necessary gconf keys shows up as: [INFO] eas_contact: backend failed: fetching folder list: GDBus.Error:org.meego.activesyncd.Error.AccountNotFound: Failed to find account [syncevolution@lists.intel.com] * Minor memory leak fix when using GDBus GIO: GDBusMethodInfo Also depends on a glib fix, see https://bugzilla.gnome.org/show_bug.cgi?id=695376 * build fixes Avoid -lrt in make dependencies. Add missing pcre libs to syncevo-dbus-server. sqlite backend needs “#include <stdio.h>” (patch from Mario Kicherer). 1.3.99.1 => 1.3.99.2, 13.12.2012 ================================ * PIM Manager searches for a caller ID (‘phone’ search) in EDS directly while folks still starts up. No unification is done of these results. Intermediate results are replaced by the final ones from folks once those are ready. * PIM Manager: allow configuration of session directories (part of [FDO #55921](https://bugs.freedesktop.org/show_bug.cgi?id=55921)) Useful for moving the session directories to a temporary file system. They are essentially just useful for debugging when used as part of PIM Manager. - “logdir” - a directory in which directories are created with debug information about sync session - “maxsessions” - number of sessions that are allowed to exist after a sync (>= 0): 0 is special and means unlimited, 1 for just the latest, etc.; old sessions are pruned heuristically (for example, keep sessions where something changed instead of some where nothing changed), so there is no hard guarantee that the last n sessions are present. * PIM Manager: write less data to disk (part of [FDO #55921](https://bugs.freedesktop.org/show_bug.cgi?id=55921)) Avoid writing config file changes to disk by enabling a new “ephemeral” mode for syncing via the PIM Manager. In this mode, config file changes are not flushed resp. discarded directly. This prevents writing to .ini files in ~/.config. The “synthesis” binfile client files are still written, but they get redirected into the session directory, which can (and should) be set to a temp file system and get deleted again quickly. Data dumps are turned off now in the configs created by the PIM Manager. * syncevo-dbus-server: use syslog instead of standard output by default * syncevo-dbus-server: command line options for controlling output and startup -d, –duration=seconds/’unlimited’ Shut down automatically when idle for this duration (default 300 seconds) -v, –verbosity=level Choose amount of output, 0 = no output, 1 = errors, 2 = info, 3 = debug; default is 1. -o, –stdout Enable printing to stdout (result of operations) and stderr (errors/info/debug). -s, –no-syslog Disable printing to syslog. -p, –start-pim Activate the PIM Manager (= unified address book) immediately. * PIM Manager: store set of active address books persistently ([FDO #56334](https://bugs.freedesktop.org/show_bug.cgi?id=56334)) Together with storing the sort order persistently, this allows restarting the daemon and have it create the same unified address book again. * PIM Manager: remove colon from valid peer UID character set ([FDO #56436](https://bugs.freedesktop.org/show_bug.cgi?id=56436)) Using the UID as part of file names gets more problematic when allowing colons. Remove that character from the API and enforce the format in the source code. * PIM Manager API: introduce contact ID and use it for reading This makes it easier for a client to fully polulate its view with contact data. Previously it could happen that due to concurrent changes in the server, a client was returned data for the same contact multiple times. A client had to detect that and re-issue read requests. * PIM Manager API: optional ViewAgent.Quiescent() ([FDO #56428](https://bugs.freedesktop.org/show_bug.cgi?id=56428)) The callback is guaranteed to be invoked once when a search has finished sending its initial results, and not sooner. This makes it possible to check whether the current data contains some contact or not. * PIM Manager: limit number of search results ([FDO #56142](https://bugs.freedesktop.org/show_bug.cgi?id=56142)) A ‘limit’ search term with a number as parameter (formatted as string) can be added to a ‘phone’ or ‘any-contains’ search term to truncate the search results after a certain number of contacts. Example: Search([[‘any-contains’, ‘Joe’], [‘limit’, ‘10’]]) => return the first 10 Joes. As with any other search, the resulting view will be updated if contact data changes. The limit must not be changed in a RefineSearch(). A ‘limit’ term may (but doesn’t have to) be given. If it is given, its value must match the value set when creating the search. This limitation simplifies the implementation and its testing. The limitation could be removed if there is sufficient demand. * PIM Manager: fix refining a search Due to not mapping the local index in the view to the parent’s index, refining only worked in views where parent and child had the same index for the contacts in the search view. * PIM Manager: fix starting when done via search When the unified address book (= FullView) was not running yet at the time when a client wanted to search it, the unified address book was not started and thus the search never returned results. * PIM Manager: fix writing contact, support photo and notes folks and EDS do not support writing properties in parallel (https://bugzilla.gnome.org/show_bug.cgi?id=652659). Must serialize setting of modified properties. * PIM Manager: fix incorrect contact removal signals in filtered view The filtered view did not check whether a parent’s removed contact was really part of the view before sending a removal signal for it. * D-Bus: missing out parameters in D-Bus introspection XML ([FDO #57292](https://bugs.freedesktop.org/show_bug.cgi?id=57292)) The problem was in the C++ D-Bus binding. If the method that gets bound to D-Bus returns a value, that value was ignored in the signature: int foo() => no out parameter It works when the method was declared as having a retval: void foo (int &result) => integer out parameter This problem existed for both the libdbus and the GIO D-Bus bindings. In SyncEvolution it affected methods like GetVersions(). * PIM Manager performance: pre-compute normalized telephone numbers Looking up by phone number spends most of its cycles in normalizing of the phone numbers in the unified address book. Instead of doing that work over and over again during the search, do it once while loading. Looking up a phone number only once does not gain from this change, it even gets slower (more memory intensive, less cache locality). Only searching multiple times becomes faster. Ultimately it would be best to store the normalized strings together with the telephone number inside EDS when the contact gets created. Work on that is in progress. * PIM Manager: improve performance of FullView sorting This fixes the hotspot during populating the FullView content: moving contacts around required copying IndividualData and thus copying complex C++ structs and strings. Storing pointers and moving those avoids that, with no lack of convenience thanks to boost::ptr_vector. Reordering also becomes faster, because the intermediate copy only needs to be of the pointers instead of the full content. * PIM Manager example: add benchmarking The new “checkpoints” split up the whole script run into pieces which are timed separately, with duration printed to stdout. In addition, tools like “perf” can be started for the duration of one phase. * EDS: fix creating databases –create-database was broken in combination with the final code in EDS 3.6 because it passed NULL for the UID to e_source_new_with_uid(), which is considered an error by the implementation of that method. Must use e_source_new() if we don’t have a UID. * fixed some memory leaks, extended tests to cover new features and bugs SyncEvolution 1.3.99.1, 25.10.2012 ================================== * workarounds for warnings from g++ 4.5 * engine: : local cache sync mode This patch introduces support for true one-way syncing (“caching”): the local datastore is meant to be an exact copy of the data on the remote side. The assumption is that no modifications are ever made locally outside of syncing. This is different from one-way sync modes, which allows local changes and only temporarily disables sending them to the remote side. Another goal of the new mode is to avoid data writes as much as possible. This new mode only works on the server side of a sync, where the engine has enough control over the data flow. Setting “sync” to: - “local-cache-incremental” will do an incremental sync (if possible) or a slow sync (otherwise). This is usually the right mode to use, and thus has “local-cache” as alias. - “local-cache-slow” will always do a slow sync. Useful for debugging or after (accidentally) making changes on the local side. An incremental sync will ignore such changes because they are not meant to happen, aren’t checked for to improve performance and thus will leave client and server out-of-sync! Both modes are recorded in the sync report of the local side. The target side is the client and records the normal “two-way” or “slow” sync modes. With the current SyncEvolution contact field list, first, middle and last name are used to find matches for contacts. For events, tasks and memos, time, summary and description are used. * HTTP proxy: useProxy=0 overrides http_* env variables Previously, if http_proxy was set, a proxy was used even if explicitly disabled. This prevented disabling the use of a proxy which only made sense in some cases, like accessing something that runs locally. Explicitly telling SyncEvolution to ignore http_proxy is necessary because it doesn’t support no_proxy. * WebDAV: auto-discovery fix With Google Contact + CardDAV the auto-discovery failed after finding the default address book, without reporting that result. * command line: implement –create/remove-database Creating a database is only possible with a chosen name. The UID is chosen automatically by the storage. Only implemented in the EDS backend. * file backend: sub-second mod time stamps Change tracking in the file backend used to be based on the modification time in seconds. When running many syncs quickly (as in testing), that can lead to changes not being detected when they happen within a second. Now the file backend also includes the sub-second part of the modification time stamp, if available. This change is relevant when upgrading SyncEvolution: most of the items will be considered “updated” once during the first sync after the upgrade (or a downgrade) because the revision strings get calculated differently. * D-Bus server: avoid progress outside of 0-100% range For example in the new TestLocalCache.testItemDelete100, the percentage value in the ProgressChanged signal become larger than 100 and then revert to 100 at the end of the sync. Seems the underlying calculation is faulty or simply inaccurate. This is not fixed. Instead the result is just clipped to the valid range. * code cleanup + improvements in testing Source, Installation, Further information ========================================= http://syncevolution.org/blogs/pohly/2013/syncevolution-13993-released Source code bundles for users are available in http://downloads.syncevolution.org/syncevolution/sources and the original source is [the git repositories](http://cgit.freedesktop.org/SyncEvolution/). i386, lpia and amd64 binaries for Debian-based distributions are available via the “unstable” syncevolution.org repository. Add the following entry to your /apt/source.list:
debhttp://downloads.syncevolution.org/aptunstablemain Then install “syncevolution-evolution”, “syncevolution-kde” and/or “syncevolution-activesync”. These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 10.04 LTS (Lucid), except for “syncevolution-activesync” which depends on libraries in Debian Squeeze, for example EDS 3.4. Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/). In contrast to 0.8.x archives, the 1.x .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps. More specific [HOWTOs](/wiki/howto) can be found in the Wiki.
Minor (or major, if you depend on auto syncing) bug fix release. Automatic syncing only ran once and notifications were not translated. * auto sync: only synced once ([FDO #56667](https://bugs.freedesktop.org/show_bug.cgi?id=56667)) A successful sync was incorrectly treated like a sync with a permanent failure, which prevents further automatic syncing. * auto sync: notifications were not translated The code which enabled localization of messages created by the D-Bus server was incomplete. Localization was only enabled accidentally through KDE if the KDE platform modules was enabled during compilation and installed. * HTTP Proxy: useProxy=0 overrides http_* env variables Previously, if http_proxy was set, a proxy was used even if explicitly disabled. This prevented disabling the use of a proxy which only made sense in some cases, like accessing something that runs locally. Explicitly telling SyncEvolution to ignore http_proxy is necessary because it doesn’t support no_proxy. * minor changes in testing and autotools files (missing Boost search path in gdbus* libs might have caused compile problems) Upgrading from release 1.2.x —————————- The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts): syncevolution –configure \ syncFormat=text/x-vcard \ mobical addressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually: syncevolution –configure \ enableRefreshSync=TRUE \ funambol Upgrading from releases before 1.2 ———————————- Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Source, Installation, Further information ========================================= Source code bundles for users are available in http://downloads.syncevolution.org/syncevolution/sources and the original source is in [the git repositories](http://cgit.freedesktop.org/SyncEvolution/). i386, lpia and amd64 binaries for Debian-based distributions are available via the “stable” syncevolution.org repository. Add the following entry to your /apt/source.list:
debhttp://downloads.syncevolution.org/aptstablemain Then install “syncevolution-evolution”, “syncevolution-kde” and/or “syncevolution-activesync”. These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy), except for “syncevolution-activesync” which depends on libraries in Debian Squeeze, for example EDS 3.4. Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/). In contrast to 0.8.x archives, the 1.x .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
SyncEvolution 1.3.1 fixes some minor issues found after 1.3 was released. * command line: fix output of –import for directories The running count at the start of the line (#0, #1, …) was not incremented when reading individual files from a directory. * Funambol: work around PHOTO TYPE=image/jpeg, part II The final version of the fix hadn’t made it into the source code. * vCalendar 1.0 + tasks: DUE date could be shifted by a day ([FDO #55238](https://bugs.freedesktop.org/show_bug.cgi?id=55238)) Because of incomplete support for time conversion, the due date could get mixed up when phone and PC were set to something other than UTC. Reported and fixed by Peter Jan. * syncevolution.org: syncevolution-evolution had incorrect dependencies Installation on older Linux distros was not possible because the ebook/ecal package dependencies were named incorrectly, for example libebook-1.2-10 instead of libebook1.2-10. Only more recent packages have the extra dash, for example libebook-1.2-12. Reported by Mariusz Sokolowski. * GTK-3 UI: fixed compile problem The GTK-3 UI depends on a class from gio-unix-2.0 and failed to compile on Fedora Core 16 because the configure checks for that lib (and thus the compiler flags) were missing. Reported by Peter Robinson. * Curl: allow using it in the D-Bus server In the past, using curl as HTTP transport in the syncevo-dbus-server was prevented, leading to “unsupported transport type is specified in the configuration”. The reason was that using curl would block the server and make it unresponsive on D-Bus. This reason has gone away, because now the HTTP traffic happens in a separate process. Thus now it is allowed to use curl in the syncevo-dbus-server. * fix for false negative in syncevo-dbus-server testing Upgrading from release 1.2.x —————————- The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts): syncevolution –configure \ syncFormat=text/x-vcard \ mobical addressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually: syncevolution –configure \ enableRefreshSync=TRUE \ funambol Upgrading from releases before 1.2 ———————————- Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Source, Installation, Further information ========================================= Source code bundles for users are available in http://downloads.syncevolution.org/syncevolution/sources and the original source is in [the git repositories](http://cgit.freedesktop.org/SyncEvolution/). i386, lpia and amd64 binaries for Debian-based distributions are available via the “stable” syncevolution.org repository. Add the following entry to your /apt/source.list:
debhttp://downloads.syncevolution.org/aptstablemain Then install “syncevolution-evolution”, “syncevolution-kde” and/or “syncevolution-activesync”. These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy), except for “syncevolution-activesync” which depends on libraries in Debian Squeeze, for example EDS 3.4. Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/). In contrast to 0.8.x archives, the 1.x .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
After almost three months of public beta testing the next major version of SyncEvolution is ready for release. The pre-releases did have the desired effect of flushing out bugs not found by nightly testing alone. Thanks everyone for packaging, downloading and testing them! Project status ============== SyncEvolution synchronizes personal information management (PIM) data via various protocols (SyncML, CalDAV/CardDAV, ActiveSync). It syncs contacts, appointments, tasks and memos. It syncs to web services or to SyncML-capable phones via Bluetooth. Binaries are available for Linux desktops (using GNOME Evolution, or KDE’s Akonadi), for MeeGo and for Maemo (Nokia N900, N9). The project moved its bug tracking from bugs.meego.com to bugs.freedesktop.org. All the old issues were migrated. Please file new issues here: http://bugs.freedesktop.org/enter_bug.cgi?product=SyncEvolution The git repositories were also moved to freedesktop.org: http://cgit.freedesktop.org/SyncEvolution/ SyncEvolution 1.3 ================= New features are KDE/Akonadi and ActiveSync support, not only in the source code but also in syncevolution.org binaries. ActiveSync is the recommended way of synchronizing contacts with Google: https://syncevolution.org/wiki/google-contacts-activesync The D-Bus server and local sync were rewritten considerably, to make the code cleaner and more robust. The CalDAV backend now also supports tasks and memos. CalDAV and CardDAV can be used in combination with a SyncML peer (“bridging SyncML and WebDAV”), thus allowing a device which only supports SyncML to talk to a WebDAV service without any intermediate storage. 1.3 contains bug fixes that were not backported to 1.2.x, so upgrading is recommended. For example, SyncEvolution 1.3 is required for Evolution 3.4, otherwise photos are not exported properly. Support for Evolution >= 3.6 is in the source code, but not in syncevolution.org binaries. Further workarounds for recent changes in Google CalDAV and Funambol One Media were added. Upgrading from release 1.2.x —————————- The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts): syncevolution –configure \ syncFormat=text/x-vcard \ mobical addressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually: syncevolution –configure \ enableRefreshSync=TRUE \ funambol Upgrading from releases before 1.2 ———————————- Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Changes 1.2.2 -> 1.3 ==================== * ActiveSync: updated to work with latest activesyncd and Google, package binaries Syncing Google contacts was added to the nightly testing. Syncing contacts and events with Exchange 2012 was already working. Setup instructions and known issues are described here: https://syncevolution.org/wiki/google-contacts-activesync * phone sync: delete<->delete conflict + phone calendar+todo sync ([BMC #23744](https://bugs.meego.com/show_bug.cgi?id=23744)) When deleting an item on phone and locally, the next sync failed with ERROR messages about “object not found”. This has several reasons: - libsynthesis super data store attempts to read items which may or may not exist (triggers ERROR message) - it checks for 404 but Evolution backends only return a generic database error (causes sync to fail) * phone sync: get phone vendor and model from Device ID profile ([BMC #736](https://bugs.meego.com/show_bug.cgi?id=736)) In the past we have relied on the user-modifiable device name to be the fingerprint for matching a phone to a template which is unreliable. This release changes this in the cases where the phone supports the Device ID profile (DIP). If support for DIP is detected, then we extract the vendor and product ids and attempt to associate them with a product and vendor name by using a newly added lookup table. This lookup table has to be maintained manually and depends on contributions by users to cover more devices. See http://blixtra.org/blog/2011/09/22/syncevolution-needs-you-or-at-least-your-bluetooth-phones/ * vCalendar 1.0: fixed recurring all-day event support vCalendar 1.0 cannot represent all-day events. The workarounds for mapping iCalendar 2.0 all-day events into vCalendar 1.0 was incomplete, leading to effects like shifting EXDATEs and end times. * Funambol: ignore UID Funambol’s OneMedia sends UID, but not RECURRENCE-ID. That becomes a problem when multiple events of the same event series are added to a backend which follows the iCalendar 2.0 standard (CalDAV, EDS, KDE), because these events all look like the master event, and there can be only one of those. SyncEvolution now strips the UID from all events coming from any Funambol server (regardless of the version). If a future Funambol server release adds support for both UID and RECURRENCE-ID, then SyncEvolution will have to be updated to take advantage of the improved server. Because the RECURRENCE-ID is also getting stripped (despite not being set at the moment), SyncEvolution should continue to work as it does now even if the server changes. It would have been nice to limit this workaround to affected Funambol server versions, but an inquiry on the Funambol mailing list didn’t get a reply, therefore SyncEvolution is playing it safe and assumes that all future Funambol releases will have the same problem. * Funambol: work around PHOTO TYPE=image/jpeg A combination of Funambol Android and Funambol server recently led to the Funambol server sending PHOTO data with TYPE=image/jpeg. This is invalid and caused EDS to reject the photo (Vladimir Elisseev, “[SyncEvolution] issues with syncing photos”). Work around the problem by only keeping the part of the type after the last slash, if there is any. For image/jpeg and similar types that leads to the desired value and does not affect valid values, because those do not contain a slash (http://www.iana.org/assignments/media-types/image/index.html). * Funambol: avoid slow syncs in refresh from server libsynthesis has traditionally implemented “refresh-from-server” as “delete local data” plus “slow” sync. This is more compatible, because some servers (like Google) do not support “refresh-from-server”. But it has the downside that the server cannot know that the client won’t send any data, and Funambol’s OneMedia now only allows one slow sync before blocking the next one for a certain period of time. This is done to prevent excessive resource usage by badly behaving clients. To accomodate both kinds of servers, the new “enableRefreshSync” sync property can be set set to explicitly allow the usage of the “refresh-from-server” sync mode. It’s off by default. The Funambol template has it turned on, existing configs must be updated manually (see upgrading comments below). * Mobical (aka Everdroid): stopped testing memo syncing Memos used to work, but now only trigger an unspecific 400 error on the server side. * GTK-UI: accept service config with a username again (BMC#23106) Suppressing configs with empty username had undesired side effects: modifying configs for direct syncing with a device incorrectly triggered the same error message, without any means of entering a username. The faulty check was removed without replacement. * GTK-UI: added GTK 3 version of UI When GTK 3 is found during compilation, a GTK 3 version of the UI is built. The source code of both is different to avoid excessive use of ifdefs. At the moment, both versions offer the same features. In the long run, the GTK 3 version will replace the GTK 2 version. * command line: added refresh/one-way-from-local/remote ([BMC #23537](https://bugs.meego.com/show_bug.cgi?id=23537)) The -from-client/server sync modes are confusing because the direction of the data exchange depends on which side acts as SyncML server or client. This release introduces new modes which use -from-local/remote instead. The statistics and messages also use these variants now. The old modes are still understood, but are declared as “not recommended” in the documentation. * command line: config and source names are optional ([BMC #23783](https://bugs.meego.com/show_bug.cgi?id=23783)) The need to add “foo” and “bar” pseudo config and source names to the command line even when all parameters for the operation where explicitly specified on the command line was confusing. Now it is possible to invoke item operations without the config and source name. Names which refer to non-existent configs are still accepted, as in previous releases. Typos are handled better by producing a detailed error report which includes (as applicable): - config doesn’t exist - source doesn’t exist or not selected - backend property not set Because luids used to be positional arguments after and , a new –luids keyword is necessary to indicate that the ensuing parameters are luids and not and . * command line: introduced –print-databases, supported for CalDAV/CardDAV Listing databases is now a dedicated operation, instead of being done whenever syncevolution was invoked without parameters. Advantages: - can be combined with property assignments for backends which do not work without that additional information, for example CalDAV/CardDAV: syncevolution –print-databases \ backend=[caldav|carddav] \ syncURL=… \ username=… \ password=… - can be done for configured sources * command line: use both stdout and stderr Traditionally, the “syncevolution” command line tool mixed its INFO/ERROR/DEBUG messages into the normal stdout. This has the major drawback that error messages get lost during operations like syncevolution –export - @default addressbook | grep “John Doe” Now anything which is not the expected result of the operation is always sent to stderr. Obviously this includes ERROR messages. INFO and DEBUG are harder to decide. Because they usually convey meta information about the running operation, they are also sent to stderr. The output of running a sync goes to both stdout (summary) and stderr (progress). * command line: allow setting empty properties Due to the way how properties were handled internally, it wasn’t possible to explicitly set a property to its default value. Instead the property was unset. For example, explicitly setting database= was not possible. This is necessary for client-test and ActiveSync, because client-test needs to know that the testing is expected to run with the default databases (something which normally is avoided by overwriting empty database properties). Now the “is set” state is tracked explicitly in the config storage and command line property APIs. Unsetting a property via the command line could be implemented with an explicit command line option, but is not supported at the moment. * command line: fixed –export When exporting items into a file, the delimiter between items was missing. * command line + local sync: fixed erroneous “Comparison impossible” output. “Comparison impossible” was incorrectly printed after a successful comparison on the target side of local sync. * local sync: fix timeout with local sync with libdbus When using libdbus instead of GIO D-Bus (as done by syncevolution.org binaries and SyncEvolution on Maemo), local sync may have aborted after 25 seconds when syncing many items with a D-Bus timeout error: [ERROR] sending message to child failed: org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible ca Reported by Toke Høiland-Jørgensen for Harmattan. Somehow not encountered elsewhere. * synccompare: shorter data dump of PHOTO A full comparison of the base64 PHOTO data can be very long. Now some key characteristics of the PHOTO data (number of characters in base64 encoding, number of bytes in decoded data, md5sum of decoded data) are printed instead. That way, unintended changes of the data (different encoding, different content) should still be found while testing and added/removed photos are nicely visible in synccompare diffs. * synccompare: fixed output for byte-identical duplicates If database dumps contained byte-identical duplicates, they were treated as a single item on the left side of a comparison. This caused erroneous “added” entries on the right side. * secure password storage: usage of GNOME Keyring vs. KDE KWallet configurable Automatically detecting KDE users is not possible at the moment. Instead KDE users have to manually set the new “keyring” global config property to “KDE” (case insensitive) if the SyncEvolution installation supports both, because GNOME Keyring is the default to avoid surprises for traditional users. If only KWallet support is enabled, then this is not necessary. “GNOME” and “true/false/1/0/yes/no” can also be set. This has the advantage that keyring usage can be enabled permanently for the command line in –daemon=no mode; normally keyrings are not used in that mode because accessing them can bring up UI dialogs. It also becomes possible to disable keyring usage in syncevo-dbus-server, something which couldn’t be done before. The –keyring command line option is still supported, as an alias for “[–sync-property] keyring=”. The default value for –keyring is true, to match the traditional behavior. In contrast to other sync properties, setting “keyring” does not require an explicit –run parameter. Again this is done to mirror traditional usage. * config: improved ‘maxlogdirs’ documentation The old explanation made it sound like nothing would get deleted by default (“If set, …”). That’s not correct, by default only 10 sessions are kept. Also explain the behavior of deleting intermediate sessions first. * Evolution: always create databases (PTCOM-113) Always try to create address book or calendar database, because even if there is a source there’s no guarantee that the actual database was created already; the original logic for only setting this when explicitly requesting a new database therefore failed in some cases. This problem affected users who had never created anything locally and wanted to use SyncEvolution to migrate their data. Now that works without having to create dummy entries first. * Evolution contacts: changed default sync format to vCard 3.0 vCard 3.0 is the better default because it has saner encoding rules and defines more properties, thus avoiding the need for non-standard extensions. However, Mobical has problems with the new default. See upgrade instructions below. * Evolution: added support for EDS 3.5.x When compiled against EDS 3.5.x or later, SyncEvolution now uses the backend code originally written for the EClient API introduced in EDS 3.2. That code was changed so that it works with the new include file rules and ESourceRegistry in EDS 3.5.x. Support for using the EClient API with EDS 3.4 was removed because maintaining three different flavors of the EDS backend code would be too much work and not gain much (just the possibility to test the EDSClient code with 3.4). At the moment, this is a compile time choice made automatically by configure. syncevolution.org binaries are compiled against an older EDS and thus do not work with EDS 3.5.x or later. EDS 3.5.x handles authentication itself, using a standard system prompt if necessary. SyncEvolution can no longer provide the password, and thus the “databaseUser/Password” options have no effect when using EDS 3.5.x. * D-Bus server: fixed HTTP presence for recent libdbus Testing with libdbus 1.6.0 on Debian Testing failed because the lib changed some behavior: instead of looking up the owner of a certain bus name immediately, it now does that when invoking a method. Therefore the check for “have connection” in SyncEvolution was too simplistic and missed the fact that both were not usable, causing the server to assume that HTTP was down while in reality it should have assumed it to be up. This prevented auto-syncing and manually clicking “Sync” in the GTK UI. * D-Bus server: made notification verbosity configurable with “notifyLevel” The new “notifyLevel” per-peer configuration option allows users to control how many desktop notifications the D-Bus server produces while executing an automatic sync: 0 - suppress all notifications 1 - show only errors 2 - show information about changes and errors (in practice currently the same as level 3) 3 - show all notifications, including starting a sync (default) * WebDAV: fixed data corruption issue when uploading item with long UID In some cases data with a very long UID wasn’t handled correctly, causing the out-going data to be malformed and probably causing a rejection by the server. The root cause is incorrect string handling. In releases before 1.2.99.1, only the –import operation of contacts into CardDAV were affected. In 1.2.99.1, the same code also got used for calendar items and then could also affect syncing. * CalDAV: updated Google workarounds Google started sending empty items (VCALENDAR with no VEVENT inside) which cannot be removed. SyncEvolution 1.3 ignores such items. The workaround for a 404 from Google Calendar for a GET (sending a REPORT request matching the item’s UID) was broken: first, processing the result ended up calling the unset responseEnd boost function pointer, which caused the request to fail. Second, getting multiple items wasn’t handled (data from all items concatenated together was used). That can happen in the somewhat unlike case that some items have a UID which is a complete superset of the requested UID - not realistic in real life, but happens during testing. * Google Calendar: updated URL redirect handling Google Calendar sometimes returns redirect requests to human-readable web sites (an “unavailable” page, a login form). This is of course bogus when the client is an automated CalDAV client. The “unavailable.html” case was already handled. Made it a bit more flexible to also catch possible variations of it (additional parameters, https instead of http). Added the https://accounts.google.com/ServiceLogin case. Not sure whether retrying will help in that case, but there’s not much else that SyncEvolution can do. * WebDAV: bridge with SyncML Now a peer accessed via SyncML can read/write data stored in a CalDAV/CardDAV server directly. This can be used to connect a device which only supports SyncML to a CalDAV/CardDAV server, or sync data between a SyncML server and a CalDAV/CardDAV server. See “CalDAV and CardDAV” in the README for details. * WebDAV: improved –configure Added INFO output about checking sources. This helps with WebDAV when the server cannot be contacted (dead, misconfigured) because otherwise there would be no indication at all why the –configure operation seems to hang. Here is some example output, including aborting:
$syncevolution--configure--templatewebdav\syncURL=http://192.168.1.100:9000/\username=foopassword=barretryDuration=2s\target-config@webdav-temp[INFO]creatingconfigurationtarget-config@webdav-temp[INFO]addressbook:lookingfordatabases...[INFO]addressbook:nodatabasetosynchronize[INFO]calendar:lookingfordatabases...[INFO]calendar:nodatabasetosynchronize[INFO]memo:lookingfordatabases...[INFO]memo:nodatabasetosynchronize[INFO]todo:lookingfordatabases...[INFO]todo:nodatabasetosynchronize It timed out fairly quickly here because of the retryDuration=2s. That also gets placed in the resulting config, which is probably not desired. Aborting the operation is now supported:
$syncevolution--configure\--templatewebdav\syncURL=http://192.168.1.100:9000/\username=foopassword=bar\target-config@webdav-temp[INFO]creatingconfigurationtarget-config@webdav-temp[INFO]addressbook:lookingfordatabases...^C[INFO]Askingtosuspend...[INFO]PressCTRL-Cagainquickly(within2s)tostopimmediately(cancauseproblemsinthefuture!)^C[INFO]Abortingimmediately...[ERROR]errorcodefromSyncEvolutionabortedonbehalfofuser(local,status20017):abortingasrequestedbyuser It would be good to make the CTRL-C handling code aware that it can abort immediately instead of doing the intermediate “asking to suspend” step, which only makes sense for sync sessions. * WebDAV: support tasks and memos ([BMC #24893](https://bugs.meego.com/show_bug.cgi?id=24893)) The new backend property values “CalDAVTodo” and “CalDAVJournal” select tasks resp. memos stored in a CalDAV collection. “CalDAV” continues to select events. Events, tasks and journals can be mixed in the same resource (= URL). However, this is less efficient than storing them separately. A good CalDAV server allows filtering items by type, and SyncEvolution uses that. However, it was found that Radicale 0.7 ignores this filtering, which could have led to data loss (SyncEvolution asks for all VTODOs in preparation for a “delete all items” operation in a “CalDAVTodo” source, gets also VJOURNALs, then deletes them). Therefore SyncEvolution plays it safe and downloads the VTODO and VJOURNAL data to double-check that it is working on the right items. This causes additional traffic for well-behaving servers; currently it cannot be turned off. Tasks are exchanged as vCalendar 1.0 or iCalendar 2.0 VJOURNAL. Memos are exchanged as VTODO or plain text. The logic for storing incoming plain text is slightly different compared to the way how the EDS memo backend did it: instead of copying the first line from the text into the summary, it is now moved. In other words, the first line gets stripped. The change is primarily technically motivated; both approaches have pros and cons. * WebDAV: improved Radicale support Radicale > 0.7 will return status 200 for delete requests; is now treated like 204 by SyncEvolution. 412 ‘Preconditiona Failed’ when asking to delete an already removed item is treated like the more common 404 ‘not found’. Same with 410 ‘gone’ instead of 404 when trying to read a non-existent item. * CalDAV/CardDAV sync: improved target side output Added a “target side of local sync ready” INFO message to introduce the output which has the target context in the [INFO] tag. The sync report from the target side now has the target context embedded in brackets after the “Changes applied during synchronization” header, to avoid ambiguities. Sometimes the backend has to resend requests because of temporary issues. If the problem turned out to be permanent, there was a long period of time, retryDuration=5 minutes to be precice, in which no visible progress happened. Now SyncEvolution’s WebDAV backend will print a message like this before going to sleep until it is time to retry: [INFO @googlecalendar] operation temporarily (?) failed, going to retry in 5.0s before giving up in 18.4s: PROPFIND: Neon error code 1: 401 Unauthorized The uncertainty comes from several factors. In this example, the 401 might indicate a permanent problem (wrong credentials), or it could be Google reporting a temporary authorization problem which is (probably) meant to slow down the client while it asks the user to re-enter the password. SyncEvolution only asks for passwords once, so it tries again with the same password if it was successful with it in the past. Otherwise it gives up immediately. Another dubious example are name server lookup errors. They can be permanent (wrong host name) or temporary (name server down). SyncEvolution errs on the side of retrying, to avoid interrupting an operation which still has a chance to continue. Output from the target side of a local sync was passed through stderr redirection as chunks of text to the frontends. This had several drawbacks: - forwarding only happened when the local sync parent was processing the output redirection, which (due to limitations of the implementation) only happens when it needs to print something itself - debug messages were not forwarded - message boundaries might have been lost In particular the new INFO messages are relevant while the sync runs and need to be shown immediately. * WebDAV: –status for WebDAV source aborted The command line –status operation did not complete when applied to a CalDAV/CardDAV source. Instead it aborted because the operation took a code path where the backend was not fully initialized. * file backend: more flexible sync support for memos The databaseFormat=text/calendar for memos did not support synchronizing as plain text. When using the new databaseFormat=text/calendar+plain, vCalendar/iCalendar/plain text are all valid sync formats; the storage is iCalendar 2.0 VJOURNAL in all cases. * WebDAV: avoid potential crash during database detection When a server responds to a PROPFIND for a path with results for some other path, then SyncEvolution crashed during the search for the default calendar or address book because of a bug in the code which was meant to handle that kind of response. Apparently Yahoo Calendar did that. Now seen again in combination with Radicale 0.6.4. In general, the code was made more robust to cope with bugs in Radicale 0.6.4. Later Radicale versions fixed these issues and also worked with SyncEvolution 1.2.2 without client-side workarounds. * WebDAV: better path normalization “syncURL” and “database” properties had to end in a trailing slash, otherwise items were not found (404 errors). Now the necessary slash is added automatically. * Curl transport: support SSLServerCertificates= When the setting refers to a directory, then CURLOPT_CAINFO doesn’t work (must be a file). Check this and use CURLOPT_CAPATH instead. Caveat: there are some comments in the API documentation about “NSS enabled libcurl” which supports a directory in CURLOPT_CAINFO. Hopefully providing an explicit path in CURLOPT_CAPATH also works in that configuration. * code cleanup + rewrite: syncing done in separate process syncevo-dbus-server now runs syncing in a separate process. Local sync also uses a second helper process. This makes the D-Bus server more responsive via D-Bus (no more blocking operations) and minimizes the effect of bugs in code involved with syncing (backends, system libraries, etc.). In the long term this restructuring will also allow more advanced features, like monitoring local or remote storage for changes. * SyncEvolution <-> SyncEvolution sync: multiple cycles per session SyncML only allows one send/receive cycle per session. There are cases (for example, client side merges data that a dumber server failed to match correctly) where client and server are still out of sync at the end of a cycle. When SyncEvolution syncs with another SyncEvolution instance (locally or remotely), both sides detect that the peer can continue syncing in the same session and start over automatically when needed. Previously the user had to start another sync session manually. To the user this is shown as “number of cycles” in a sync session in the sync report. “Restart” is the process of entering a new cycle. The cycles are also visible in the command line output as multiple INFO lines:
`
[INFO] eds_contact: starting first time sync from client (peer is server)
[INFO] creating complete data backup of source eds_contact before sync (enabled with dumpData and needed for prin
Local data changes to be applied during synchronization:
*** eds_contact ***
no changes
Work towards the 1.3 release of SyncEvolution continued with two more snapshots since the last announcement. The project also moved its bug tracking from bugs.meego.com to bugs.freedesktop.org. Most of the old issues were already migrated, the rest will follow later. Please [file new issues](http://bugs.freedesktop.org/enter_bug.cgi?product=SyncEvolution) there. The git repos are still at meego.com; they will move to freedesktop.org eventually. SyncEvolution 1.2.99.3 -> 1.2.99.4, 07.08.2012 ============================================== Another release candidate for SyncEvolution 1.3. Lesson learned: declaring a snapshot as “final” is a good way of luring the hidden bugs into the light. Of course, then another snapshot is needed… Details: * D-Bus server: fix support for autoSyncDelay > 0 Auto syncing was not getting triggered when using an autoSyncDelay > 0; by default it is 5 minutes. Thanks to Vladimir Elisseev for reporting this problem. * command line: fixed –export When exporting items into a file, the delimiter between items was missing. * config: improved ‘maxlogdirs’ documentation The old explanation made it sound like nothing would get deleted by default (“If set, …”). That’s not correct, by default only 10 sessions are kept. Also explain the behavior of deleting intermediate sessions first. * developers: fixed D-Bus interface XML Reverted to Qt 4.x compatible annotations and changed “templateName” to “getTemplate” to make it more obvious what the parameter does. Only relevant for the out-of-tree Qt UI. Fixed accidental removal of the “template” parameter in Session.GetNamedConfig(). Was not used in practice, but has to be correct in case that someone wants to use it. SyncEvolution 1.2.99.2 -> 1.2.99.3, 24.07.2012 ============================================== Final release candidate for SyncEvolution 1.3 - fingers crossed, knock on wood, etc. ActiveSync is now available in binaries from syncevolution.org and becomes the recommended way of synchronizing contacts with Google. EDS 3.5.x and later are supported when compiling from source; syncevolution.org binaries continue to support only EDS up to 3.4. Details: * EDS: added support for EDS 3.5.x When compiled against EDS 3.5.x or later, SyncEvolution now uses the backend code originally written for the EClient API introduced in EDS 3.2. That code was changed so that it works with the new include file rules and ESourceRegistry in EDS 3.5.x. Support for using the EClient API with EDS 3.4 was removed because maintaining three different flavors of the EDS backend code would be too much work and not gain much (just the possibility to test the EDSClient code with 3.4). At the moment, this is a compile time choice made automatically by configure. syncevolution.org binaries are compiled against an older EDS and thus do not work with EDS 3.5.x or later. EDS 3.5.x handles authentication itself, using a standard system prompt if necessary. SyncEvolution can no longer provide the password, and thus the “databaseUser/Password” options have no effect when using EDS 3.5.x. * ActiveSync: updated to work with latest activesyncd and Google, package binaries Syncing Google contacts was added to the nightly testing. Syncing contacts and events with Exchange 2012 was already working. Setup instructions and known issues are described here: https://syncevolution.org/wiki/google-contacts-activesync * local sync: don’t drop data comparison output on target side synccompare on the target side of a local sync was invoked with its output being redirected via an unreliable socket to the local sync parent. When the output was large, some of it might have been lost. * local sync: fixed crash When processing stdout from syncevo-local-child in syncevo-dbus-helper, the LogRedirect class was invoked recursively and tried to print the same stdout data repeatedly until the syncevo-dbus-helper crashed due to the infinite recurssion. * local sync: fixed helper process shutdown in case of parent failure The helper process only detected that the parent failed when it tried to log something while the parent had already shut down the D-Bus connection. Even that did not work reliably and differed between D-Bus libdbus and GIO. Added several test cases and fixes for “process died prematurely” error scenarios. * Mobical (aka Everdroid): stopped testing memo syncing Memos used to work, but now only trigger an unspecific 400 error on the server side. * autotools: ensure that link lines are complete As mentioned by Tino Keitel on the mailing list, some libs and executables were only implicitly linked against libraries that they called directly. This happened to work by chance because these libraries ended up in the running executable anyway, due to indirect loading. Now there is a “make installcheck” test for this kind of defect and the makefiles were updated to avoid it. One exception is libsmltk, which depends on the caller providing SySync logging support. * D-Bus server: fixed HTTP presence for recent libdbus Testing with libdbus 1.6.0 on Debian Testing failed because the lib changed some behavior: instead of looking up the owner of a certain bus name immediately, it now does that when invoking a method. Therefore the check for “have connection” in SyncEvolution was too simplistic and missed the fact that both were not usable, causing the server to assume that HTTP was down while in reality it should have assumed it to be up. This prevented auto-syncing and manually clicking “Sync” in the GTK UI. * syncevolution.org: declare dependencies on libical and EDS Let the bundle .deb depend on libical if the lib was enabled during compilation (for example, for CalDAV). This ensures that it gets installed on systems which otherwise don’t have it. “syncevolution-evolution” is compatible (and depends on) EDS up to and including 3.4. The package now declares that dependency and conflicts with more recent EDS, because even if the older EDS libs are still installed they won’t work when the rest of EDS was updated. * CalDAV + syncevolution.org: fixed segfault without libical+libecal When libical and libecal were not installed, trying to use the CalDAV backend for VEVENTs segfaulted because it depends on libical and did not check properly for it. Only affected syncevolution.org binaries. Upgrading from release 1.2.x: The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts): syncevolution –configure \ syncFormat=text/x-vcard \ mobical addressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually: syncevolution –configure \ enableRefreshSync=TRUE \ funambol Upgrading from releases before 1.2: Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Source, Installation, Further information =============================== Source snapshots are in http://downloads.syncevolution.org/syncevolution/sources i386, lpia and amd64 binaries for Debian-based distributions are available via the “unstable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:
debhttp://downloads.syncevolution.org/aptunstablemain These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/evolution). In contrast to 0.8.x archives, the new .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
Final release candidate for SyncEvolution 1.3 - fingers crossed, knock on wood, etc. ActiveSync is now available in binaries from syncevolution.org and becomes the recommended way of [synchronizing contacts with Google](/wiki/google-contacts-activesync). EDS 3.5.x and later are supported when compiling from source; syncevolution.org binaries continue to support only EDS up to 3.4. Details: * EDS: added support for EDS 3.5.x When compiled against EDS 3.5.x or later, SyncEvolution now uses the backend code originally written for the EClient API introduced in EDS 3.2. That code was changed so that it works with the new include file rules and ESourceRegistry in EDS 3.5.x. Support for using the EClient API with EDS 3.4 was removed because maintaining three different flavors of the EDS backend code would be too much work and not gain much (just the possibility to test the EDSClient code with 3.4). At the moment, this is a compile time choice made automatically by configure. syncevolution.org binaries are compiled against an older EDS and thus do not work with EDS 3.5.x or later. EDS 3.5.x handles authentication itself, using a standard system prompt if necessary. SyncEvolution can no longer provide the password, and thus the “databaseUser/Password” options have no effect when using EDS 3.5.x. * ActiveSync: updated to work with latest activesyncd and Google, package binaries Syncing Google contacts was added to the nightly testing. Syncing contacts and events with Exchange 2012 was already working. Setup instructions and known issues are described here: https://syncevolution.org/wiki/google-contacts-activesync * local sync: don’t drop data comparison output on target side synccompare on the target side of a local sync was invoked with its output being redirected via an unreliable socket to the local sync parent. When the output was large, some of it might have been lost. * local sync: fixed crash When processing stdout from syncevo-local-child in syncevo-dbus-helper, the LogRedirect class was invoked recursively and tried to print the same stdout data repeatedly until the syncevo-dbus-helper crashed due to the infinite recurssion. * local sync: fixed helper process shutdown in case of parent failure The helper process only detected that the parent failed when it tried to log something while the parent had already shut down the D-Bus connection. Even that did not work reliably and differed between D-Bus libdbus and GIO. Added several test cases and fixes for “process died prematurely” error scenarios. * Mobical (aka Everdroid): stopped testing memo syncing Memos used to work, but now only trigger an unspecific 400 error on the server side. * autotools: ensure that link lines are complete As mentioned by Tino Keitel on the mailing list, some libs and executables were only implicitly linked against libraries that they called directly. This happened to work by chance because these libraries ended up in the running executable anyway, due to indirect loading. Now there is a “make installcheck” test for this kind of defect and the makefiles were updated to avoid it. One exception is libsmltk, which depends on the caller providing SySync logging support. * D-Bus server: fixed HTTP presence for recent libdbus Testing with libdbus 1.6.0 on Debian Testing failed because the lib changed some behavior: instead of looking up the owner of a certain bus name immediately, it now does that when invoking a method. Therefore the check for “have connection” in SyncEvolution was too simplistic and missed the fact that both were not usable, causing the server to assume that HTTP was down while in reality it should have assumed it to be up. This prevented auto-syncing and manually clicking “Sync” in the GTK UI. * syncevolution.org: declare dependencies on libical and EDS Let the bundle .deb depend on libical if the lib was enabled during compilation (for example, for CalDAV). This ensures that it gets installed on systems which otherwise don’t have it. “syncevolution-evolution” is compatible (and depends on) EDS up to and including 3.4. The package now declares that dependency and conflicts with more recent EDS, because even if the older EDS libs are still installed they won’t work when the rest of EDS was updated. * CalDAV + syncevolution.org: fixed segfault without libical+libecal When libical and libecal were not installed, trying to use the CalDAV backend for VEVENTs segfaulted because it depends on libical and did not check properly for it. Only affected syncevolution.org binaries. Upgrading from release 1.2.x: ——————————————- The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts):
syncevolution--configure\syncFormat=text/x-vcard\mobicaladdressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually:
syncevolution--configure\enableRefreshSync=TRUE\funambol Upgrading from releases before 1.2: ————————————————— Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Source, Installation, Further information =============================== Source snapshots are in http://downloads.syncevolution.org/syncevolution/sources i386, lpia and amd64 binaries for Debian-based distributions are available via the “unstable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:
debhttp://downloads.syncevolution.org/aptunstablemain These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/evolution). In contrast to 0.8.x archives, the new .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
Next step towards SyncEvolution 1.3. It adds a workaround for Funambol’s OneMedia and fixes an old bug which became more severe in 1.2.99.1. Also has some usability improvements for CalDAV/CardDAV. Hopefully it will not take long to stabilize the code, so test it now while it is still hot :-) Details: * Funambol: ignore UID Funambol’s OneMedia sends UID, but not RECURRENCE-ID. That becomes a problem when multiple events of the same event series are added to a backend which follows the iCalendar 2.0 standard (CalDAV, EDS, KDE), because these events all look like the master event, and there can be only one of those. SyncEvolution now strips the UID from all events coming from any Funambol server (regardless of the version). If a future Funambol server release adds support for both UID and RECURRENCE-ID, then SyncEvolution will have to be updated to take advantage of the improved server. Because the RECURRENCE-ID is also getting stripped (despite not being set at the moment), SyncEvolution should continue to work as it does now even if the server changes. It would have been nice to limit this workaround to affected Funambol server versions, but an inquiry on the Funambol mailing list didn’t get a reply, therefore SyncEvolution is playing it safe and assumes that all future Funambol releases will have the same problem. * WebDAV: fixed data corruption issue when uploading item with long UID In some cases data with a very long UID wasn’t handled correctly, causing the out-going data to be malformed and probably causing a rejection by the server. The root cause is incorrect string handling. In releases before 1.2.99.1, only the –import operation of contacts into CardDAV were affected. In 1.2.99.1, the same code also got used for calendar items and then could also affect syncing. * engine: add DTSTAMP+LAST-MODIFIED before writing calendar items When writing calendar items into a backend storage as iCalendar 2.0 or vCalendar 1.0, they should have DTSTAMP and LAST-MODIFIED values. DTSTAMP is expected by some CalDAV servers (like Apple). LAST-MODIFIED is usually added by the storage, but not always. In the text/plain -> syncevolution -> text/calendar -> Radicale -> EDS -> syncevolution chain the LAST-MODIFIED was not added by Radicale, which caused problems for change tracking in an EDS-based SyncEvolution. Also necessary when importing from a phone using vCalendar without DTSTAMP directly into CalDAV. * Google Calendar: updated URL redirect handling Google Calendar sometimes returns redirect requests to human-readable web sites (an “unavailable” page, a login form). This is of course bogus when the client is an automated CalDAV client. The “unavailable.html” case was already handled. Made it a bit more flexible to also catch possible variations of it (additional parameters, https instead of http). Added the https://accounts.google.com/ServiceLogin case. Not sure whether retrying will help in that case, but there’s not much else that SyncEvolution can do. * CalDAV + VJOURNAL: handle UID conflicts When asked to insert a VJOURNAL which already existed (= same UID), CalDAV servers respond with a 412 “Precondition failed” error. This needs to be detected and translated into an “item needs to be merged” result so that the engine can load the existing item, merge the data, and then write back. * WebDAV: –status for WebDAV source aborted The command line –status operation did not complete when applied to a CalDAV/CardDAV source. Instead it aborted because the operation took a code path where the backend was not fully initialized. * CalDAV/CardDAV sync: improved target side output Added a “target side of local sync ready” INFO message to introduce the output which has the target context in the [INFO] tag. The sync report from the target side now has the target context embedded in brackets after the “Changes applied during synchronization” header, to avoid ambiguities. Sometimes the backend has to resend requests because of temporary issues. If the problem turned out to be permanent, there was a long period of time, retryDuration=5 minutes to be precice, in which no visible progress happened. Now SyncEvolution’s WebDAV backend will print a message like this before going to sleep until it is time to retry: [INFO @googlecalendar] operation temporarily (?) failed, going to retry in 5.0s before giving up in 18.4s: PROPFIND: Neon error code 1: 401 Unauthorized The uncertainty comes from several factors. In this example, the 401 might indicate a permanent problem (wrong credentials), or it could be Google reporting a temporary authorization problem which is (probably) meant to slow down the client while it asks the user to re-enter the password. SyncEvolution only asks for passwords once, so it tries again with the same password if it was successful with it in the past. Otherwise it gives up immediately. Another dubious example are name server lookup errors. They can be permanent (wrong host name) or temporary (name server down). SyncEvolution errs on the side of retrying, to avoid interrupting an operation which still has a chance to continue. Output from the target side of a local sync was passed through stderr redirection as chunks of text to the frontends. This had several drawbacks: - forwarding only happened when the local sync parent was processing the output redirection, which (due to limitations of the implementation) only happens when it needs to print something itself - debug messages were not forwarded - message boundaries might have been lost In particular the new INFO messages are relevant while the sync runs and need to be shown immediately. * command line: fixed password + property lookup during –print-databases –print-databases for an existing configuration did not look up passwords stored in a keyring, causing the operation to fail for backends like CalDAV/CardDAV where credentials are required. Overriding source properties in that case also only worked when using the unqualified property name (“databasePassword=foo”) but not when using the source name as prefix (“calendar/databasePassword=foo”). * Developers: Implementing the improved local sync output required extending the D-Bus API. The Server.LogOutput signal now has an additional “process name” parameter. Normally it is empty. For messages originating from the target side, it carries that extra target context string. This D-Bus API change is backward compatible. Older clients can still subscribe to and decode the LogOutput messages, they’ll simply ignore the extra parameter. Newer clients expecting that extra parameter won’t work with an older D-Bus daemon: they’ll fail to decode the D-Bus message. Upgrading from release 1.2.x: ——————————————- The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts):
syncevolution--configure\syncFormat=text/x-vcard\mobicaladdressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually:
syncevolution--configure\enableRefreshSync=TRUE\funambol Upgrading from releases before 1.2: ————————————————— Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Source, Installation, Further information =============================== Source snapshots are in http://downloads.syncevolution.org/syncevolution/sources i386, lpia and amd64 binaries for Debian-based distributions are available via the “unstable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:
debhttp://downloads.syncevolution.org/aptunstablemain These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/evolution). In contrast to 0.8.x archives, the new .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
First pre-release of SyncEvolution 1.3. Contains bug fixes that were not backported to 1.2.x, so upgrading is recommended. For example, SyncEvolution 1.3 is required for Evolution 3.4, otherwise photos are not exported properly. Further workarounds for recent changes in Google CalDAV were added. Major new features are KDE/Akonadi support in the syncevolution.org binaries and ActiveSync support (only in the source code). The D-Bus server and local sync were rewritten considerably, to make the code cleaner and more robust. The CalDAV backend now also supports tasks and memos. Details: * phone sync: delete<->delete conflict + phone calendar+todo sync ([BMC #23744](https://bugs.meego.com/show_bug.cgi?id=23744)) When deleting an item on phone and locally, the next sync failed with ERROR messages about “object not found”. This has several reasons: - libsynthesis super data store attempts to read items which may or may not exist (triggers ERROR message) - it checks for 404 but Evolution backends only return a generic database error (causes sync to fail) * phone sync: get phone vendor and model from Device ID profile ([BMC #736](https://bugs.meego.com/show_bug.cgi?id=736)) In the past we have relied on the user-modifiable device name to be the fingerprint for matching a phone to a template which is unreliable. This release changes this in the cases where the phone supports the Device ID profile (DIP). If support for DIP is detected, then we extract the vendor and product ids and attempt to associate them with a product and vendor name by using a newly added lookup table. This lookup table has to be maintained manually and depends on contributions by users to cover more devices. See http://blixtra.org/blog/2011/09/22/syncevolution-needs-you-or-at-least-your-bluetooth-phones/ * vCalendar 1.0: fixed recurring all-day event support vCalendar 1.0 cannot represent all-day events. The workarounds for mapping iCalendar 2.0 all-day events into vCalendar 1.0 was incomplete, leading to effects like shifting EXDATEs and end times. * GTK-UI: accept service config with a username again (BMC#23106) Suppressing configs with empty username had undesired side effects: modifying configs for direct syncing with a device incorrectly triggered the same error message, without any means of entering a username. The faulty check was removed without replacement. * GTK-UI: added GTK 3 version of UI When GTK 3 is found during compilation, a GTK 3 version of the UI is built. The source code of both is different to avoid excessive use of ifdefs. At the moment, both versions offer the same features. In the long run, the GTK 3 version will replace the GTK 2 version. * command line: added refresh/one-way-from-local/remote ([BMC #23537](https://bugs.meego.com/show_bug.cgi?id=23537)) The -from-client/server sync modes are confusing because the direction of the data exchange depends on which side acts as SyncML server or client. This release introduces new modes which use -from-local/remote instead. The statistics and messages also use these variants now. The old modes are still understood, but are declared as “not recommended” in the documentation. * command line: config and source names are optional ([BMC #23783](https://bugs.meego.com/show_bug.cgi?id=23783)) The need to add “foo” and “bar” pseudo config and source names to the command line even when all parameters for the operation where explicitly specified on the command line was confusing. Now it is possible to invoke item operations without the config and source name. Names which refer to non-existent configs are still accepted, as in previous releases. Typos are handled better by producing a detailed error report which includes (as applicable): - config doesn’t exist - source doesn’t exist or not selected - backend property not set Because luids used to be positional arguments after and , a new –luids keyword is necessary to indicate that the ensuing parameters are luids and not and . * command line: introduced –print-databases, supported for CalDAV/CardDAV Listing databases is now a dedicated operation, instead of being done whenever syncevolution was invoked without parameters. Advantages: - can be combined with property assignments for backends which do not work without that additional information, for example CalDAV/CardDAV:
-canbedoneforconfiguredsources\*commandline:usebothstdoutandstderrTraditionally,the"syncevolution"commandlinetoolmixeditsINFO/ERROR/DEBUGmessagesintothenormalstdout.Thishasthemajordrawbackthaterrormessagesgetlostduringoperationslikesyncevolution--export-@defaultaddressbook|grep"JohnDoe"Nowanythingwhichnottheexpectedresultoftheoperationisalwayssenttostderr.ObviouslythisincludesERRORmessages.INFOandDEBUGarehardertodecide.Becausetheyusuallyconveymetainformationabouttherunningoperation,theyarealsosenttostderr.Theoutputofrunningasyncgoestobothstdout(summary)andstderr(progress).\*commandline:allowsettingemptypropertiesDuetothewayhowpropertieswerehandledinternally,itwasn'tpossibletoexplicitlysetapropertytoitsdefaultvalue.Insteadthepropertywasunset.Forexample,explicitlysettingdatabase=wasnotpossible.Thisisnecessaryforclient-testandActiveSync,becauseclient-testneedstoknowthatthetestingisexpectedtorunwiththedefaultdatabases(somethingwhichnormallyisavoidedbyoverwritingemptydatabaseproperties).Nowthe"isset"stateistrackedexplicitlyintheconfigstorageandcommandlinepropertyAPIs.Unsettingapropertyviathecommandlinecouldbeimplementedwithanexplicitcommandlineoption,butisnotsupportedatthemoment.\*commandline+localsync:fixederroneous"Comparisonimpossible"output."Comparisonimpossible"wasincorrectlyprintedafterasuccessfulcomparisononthetargetsideoflocalsync.\*synccompare:shorterdatadumpofPHOTOAfullcomparisonofthebase64PHOTOdatacanbeverylong.NowsomekeycharacteristicsofthePHOTOdata(numberofcharactersinbase64encoding,numberofbytesindecodeddata,md5sumofdecodeddata)areprintedinstead.Thatway,unintendedchangesofthedata(differentencoding,differentcontent)shouldstillbefoundwhiletestingandadded/removedphotosarenicelyvisibleinsynccomparediffs.\*synccompare:fixedoutputforbyte-identicalduplicatesIfdatabasedumpscontainedbyte-identicalduplicates,theyweretreatedasasingleitemontheleftsideofacomparison.Thiscausederroneous"added"entriesontherightside.\*securepasswordstorage:usageofGNOMEKeyringvs.KDEKWalletconfigurableAutomaticallydetectingKDEusersisnotpossibleatthemoment.InsteadKDEusershavetomanuallysetthenew"keyring"globalconfigpropertyto"KDE"(caseinsensitive)iftheSyncEvolutioninstallationsupportsboth,becauseGNOMEKeyringisthedefaulttoavoidsurprisesfortraditionalusers.IfonlyKWalletsupportisenabled,thenthisisnotnecessary."GNOME"and"true/false/1/0/yes/no"canalsobeset.Thishastheadvantagethatkeyringusagecanbeenabledpermanentlyforthecommandlinein--daemon=nomode;normallykeyringsarenotusedinthatmodebecauseaccessingthemcanbringupUIdialogs.Italsobecomespossibletodisablekeyringusageinsyncevo-dbus-server,somethingwhichcouldn'tbedonebefore.The--keyringcommandlineoptionisstillsupported,asanaliasfor"[--sync-property]keyring=<value>".Thedefaultvaluefor--keyringistrue,tomatchthetraditionalbehavior.Incontrasttoothersyncproperties,setting"keyring"doesnotrequireanexplicit--runparameter.Againthisisdonetomirrortraditionalusage.\*Evolution:alwayscreatedatabases(PTCOM-113)Alwaystrytocreateaddressbookorcalendardatabase,becauseevenifthereisasourcethere'snoguaranteethattheactualdatabasewascreatedalready;theoriginallogicforonlysettingthiswhenexplicitlyrequestinganewdatabasethereforefailedinsomecases.ThisproblemaffecteduserswhohadnevercreatedanythinglocallyandwantedtouseSyncEvolutiontomigratetheirdata.Nowthatworkswithouthavingtocreatedummyentriesfirst.\*Evolutioncontacts:changeddefaultsyncformattovCard3.0vCard3.0isthebetterdefaultbecauseithassanerencodingrulesanddefinesmoreproperties,thusavoidingtheneedfornon-standardextensions.However,Mobicalhasproblemswiththenewdefault.Seeupgradeinstructionsbelow.\*D-Busserver:madenotificationverbosityconfigurablewith"notifyLevel"Thenew"notifyLevel"per-peerconfigurationoptionallowsuserstocontrolhowmanydesktopnotificationstheD-Busserverproduceswhileexecutinganautomaticsync:0-suppressallnotifications1-showonlyerrors2-showinformationaboutchangesanderrors(inpracticecurrentlythesameaslevel3)3-showallnotifications,includingstartingasync(default)\*CalDAV:updatedGoogleworkaroundsGooglestartedsendingemptyitems(VCALENDARwithnoVEVENTinside)whichcannotberemoved.SyncEvolution1.3ignoressuchitems.Theworkaroundfora404fromGoogleCalendarforaGET(sendingaREPORTrequestmatchingtheitem'sUID)wasbroken:first,processingtheresultendedupcallingtheunsetresponseEndboostfunctionpointer,whichcausedtherequesttofail.Second,gettingmultipleitemswasn'thandled(datafromallitemsconcatenatedtogetherwasused).ThatcanhappeninthesomewhatunlikecasethatsomeitemshaveaUIDwhichisacompletesupersetoftherequestedUID-notrealisticinreallife,buthappensduringtesting.\*WebDAV:bridgewithSyncMLNowapeeraccessedviaSyncMLcanread/writedatastoredinaCalDAV/CardDAVserverdirectly.ThiscanbeusedtoconnectadevicewhichonlysupportsSyncMLtoaCalDAV/CardDAVserver,orsyncdatabetweenaSyncMLserverandaCalDAV/CardDAVserver.See"CalDAVandCardDAV"intheREADMEfordetails.\*WebDAV:improved--configureAddedINFOoutputaboutcheckingsources.ThishelpswithWebDAVwhentheservercannotbecontacted(dead,misconfigured)becauseotherwisetherewouldbenoindicationatallwhythe--configureoperationseemstohang.Hereissomeexampleoutput,includingaborting:
$ syncevolution –configure –template webdav
syncURL=http://192.168.1.100:9000/
username=foo password=bar retryDuration=2s
target-config@webdav-temp
[INFO] creating configuration target-config@webdav-temp
[INFO] addressbook: looking for databases…
[INFO] addressbook: no database to synchronize
[INFO] calendar: looking for databases…
[INFO] calendar: no database to synchronize
[INFO] memo: looking for databases…
[INFO] memo: no database to synchronize
[INFO] todo: looking for databases…
[INFO] todo: no database to synchronize
IttimedoutfairlyquicklyherebecauseoftheretryDuration=2s.Thatalsogetsplacedintheresultingconfig,whichisprobablynotdesired.Abortingtheoperationisnowsupported:
$ syncevolution –configure
–template webdav
syncURL=http://192.168.1.100:9000/
username=foo password=bar
target-config@webdav-temp
[INFO] creating configuration target-config@webdav-temp
[INFO] addressbook: looking for databases…
^C[INFO] Asking to suspend…
[INFO] Press CTRL-C again quickly (within 2s) to stop immediately (can cause problems in the future!)
^C[INFO] Aborting immediately …
[ERROR] error code from SyncEvolution aborted on behalf of user (local, status 20017): aborting as requested by user
ItwouldbegoodtomaketheCTRL-Chandlingcodeawarethatitcanabortimmediatelyinsteadofdoingtheintermediate"askingtosuspend"step,whichonlymakessenseforsyncsessions.\*WebDAV:supporttasksandmemos(\[BMC#24893\](https://bugs.meego.com/show_bug.cgi?id=24893))Thenewbackendpropertyvalues"CalDAVTodo"and"CalDAVJournal"selecttasksresp.memosstoredinaCalDAVcollection."CalDAV"continuestoselectevents.Events,tasksandjournalscanbemixedinthesameresource(=URL).However,thisislessefficientthanstoringthemseparately.AgoodCalDAVserverallowsfilteringitemsbytype,andSyncEvolutionusesthat.However,itwasfoundthatRadicale0.7ignoresthisfiltering,whichcouldhaveledtodataloss(SyncEvolutionasksforallVTODOsinpreparationfora"deleteallitems"operationina"CalDAVTodo"source,getsalsoVJOURNALs,thendeletesthem).ThereforeSyncEvolutionplaysitsafeanddownloadstheVTODOandVJOURNALdatatodouble-checkthatitisworkingontherightitems.Thiscausesadditionaltrafficforwell-behavingservers;currentlyitcannotbeturnedoff.TasksareexchangedasvCalendar1.0oriCalendar2.0VJOURNAL.MemosareexchangedasVTODOorplaintext.ThelogicforstoringincomingplaintextisslightlydifferentcomparedtothewayhowtheEDSmemobackenddidit:insteadofcopyingthefirstlinefromthetextintothesummary,itisnowmoved.Inotherwords,thefirstlinegetsstripped.Thechangeisprimarilytechnicallymotivated;bothapproacheshaveprosandcons.\*WebDAV:improvedRadicalesupportRadicale>0.7willreturnstatus200fordeleterequests;isnowtreatedlike204bySyncEvolution.412'PreconditionaFailed'whenaskingtodeleteanalreadyremoveditemistreatedlikethemorecommon404'notfound'.Samewith410'gone'insteadof404whentryingtoreadanon-existentitem.\*filebackend:moreflexiblesyncsupportformemosThedatabaseFormat=text/calendarformemosdidnotsupportsynchronizingasplaintext.WhenusingthenewdatabaseFormat=text/calendar+plain,vCalendar/iCalendar/plaintextareallvalidsyncformats;thestorageisiCalendar2.0VJOURNALinallcases.\*WebDAV:avoidpotentialcrashduringdatabasedetectionWhenaserverrespondstoaPROPFINDforapathwithresultsforsomeotherpath,thenSyncEvolutioncrashedduringthesearchforthedefaultcalendaroraddressbookbecauseofabuginthecodewhichwasmeanttohandlethatkindofresponse.ApparentlyYahooCalendardidthat.NowseenagainincombinationwithRadicale0.6.4.Ingeneral,thecodewasmademorerobusttocopewithbugsinRadicale0.6.4.LaterRadicaleversionsfixedtheseissuesandalsoworkedwithSyncEvolution1.2.2withoutclient-sideworkarounds.\*WebDAV:betterpathnormalization"syncURL"and"database"propertieshadtoendinatrailingslash,otherwiseitemswerenotfound(404errors).Nowthenecessaryslashisaddedautomatically.\*Funambol:avoidslowsyncsinrefreshfromserverlibsynthesishastraditionallyimplemented"refresh-from-server"as"deletelocaldata"plus"slow"sync.Thisismorecompatible,becausesomeservers(likeGoogle)donotsupport"refresh-from-server".Butithasthedownsidethattheservercannotknowthattheclientwon'tsendanydata,andFunambol'sOneMedianowonlyallowsoneslowsyncbeforeblockingthenextoneforacertainperiodoftime.Thisisdonetopreventexcessiveresourceusagebybadlybehavingclients.Toaccomodatebothkindsofservers,thenew"enableRefreshSync"syncpropertycanbesetsettoexplicitlyallowtheusageofthe"refresh-from-server"syncmode.It'soffbydefault.TheFunamboltemplatehasitturnedon,existingconfigsmustbeupdatedmanually(seeupgradingcommentsbelow).*Curltransport:supportSSLServerCertificates=<path>Whenthesettingreferstoadirectory,thenCURLOPT\_CAINFOdoesn'twork(mustbeafile).CheckthisanduseCURLOPT\_CAPATHinstead.Caveat:therearesomecommentsintheAPIdocumentationabout"NSSenabledlibcurl"whichsupportsadirectoryinCURLOPT\_CAINFO.HopefullyprovidinganexplicitpathinCURLOPT\_CAPATHalsoworksinthatconfiguration.\*codecleanup+rewrite:syncingdoneinseparateprocesssyncevo-dbus-servernowrunssyncinginaseparateprocess.Localsyncalsousesasecondhelperprocess.ThismakestheD-BusservermoreresponsiveviaD-Bus(nomoreblockingoperations)andminimizestheeffectofbugsincodeinvolvedwithsyncing(backends,systemlibraries,etc.).Inthelongtermthisrestructuringwillalsoallowmoreadvancedfeatures,likemonitoringlocalorremotestorageforchanges.\*SyncEvolution<->SyncEvolutionsync:multiplecyclespersessionSyncMLonlyallowsonesend/receivecyclepersession.Therearecases(forexample,clientsidemergesdatathatadumberserverfailedtomatchcorrectly)whereclientandserverarestilloutofsyncattheendofacycle.WhenSyncEvolutionsyncswithanotherSyncEvolutioninstance(locallyorremotely),bothsidesdetectthatthepeercancontinuesyncinginthesamesessionandstartoverautomaticallywhenneeded.Previouslytheuserhadtostartanothersyncsessionmanually.Totheuserthisisshownas"numberofcycles"inasyncsessioninthesyncreport."Restart"istheprocessofenteringanewcycle.ThecyclesarealsovisibleinthecommandlineoutputasmultipleINFOlines:
[INFO] eds_contact: starting first time sync from client (peer is server)
[INFO] creating complete data backup of source eds_contact before sync (enabled with dumpData and needed for prin
Local data changes to be applied during synchronization:
*** eds_contact ***
no changes
Maintenance release with various bug fixes. * syncevo-dbus-server + ConnMan: fixed “online” detection ([BMC #21541](https://bugs.meego.com/show_bug.cgi?id=21541), [BMC #24587](https://bugs.meego.com/show_bug.cgi?id=24587)) SyncEvolution did not recognize any cellular connectivity as suitable for syncing. The strict check for certain “connected technology” is unnecessary, anything which makes the computer “online” should be good enough. So now it just uses the ConnMan “State” property. Additional benefit: will continue to work with ConnMan 1.0, which won’t have the “ConnectedTechnologies” property anymore. The Bluetooth available check was also (incorrectly) using the ConnMan API. Now asssume that OBEX/Bluetooth is always available. * automatic backups: added INFO messages and fixed dumpData/printChanges ([BMC #24619](https://bugs.meego.com/show_bug.cgi?id=24619)) Point out that backups are created (user might be unaware otherwise and wonder about the delay), explain why (so that users know how to turn it off). Turning these backups off with dumpData=0 printChanges=0 had to be fixed, backups were always written previously. * EDS compatibility: bumped version check for EDS 3.2 SyncEvolution is known to work with EDS 3.2. Therefore use the libebook/ecal/edataserver libs from 3.2 if available, without warnings in the –version output. Also happens with inconsistent distro setups where the old libs are available and would have been prefered by SyncEvolution 1.2.1 even though the old libs no longer work with EDS 3.2. * GTK-UI: do not accept service config without a username ([BMC #23106](https://bugs.meego.com/show_bug.cgi?id=23106)) Instead of creating such a config, an error dialog is shown. * GTK-UI: updated translations * fixed various compile issues, primarily on Fedora Core 17 (unistd.h/ssize_t, invoking syncevolution during compilation, missing src/dbus/qt/configure-sub.in) SyncEvolution is known to not compile with Bluez 4.97. A patch for Bluez header files is needed to make them work in C++ again, see http://thread.gmane.org/gmane.linux.bluez.kernel/20364 Upgrading from releases before 1.2 ————————————————– Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Release 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Source, Installation, Further information =============================== Source snapshots are in http://downloads.syncevolution.org/syncevolution/sources i386, lpia and amd64 binaries for Debian-based distributions are available via the “stable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:
debhttp://downloads.syncevolution.org/aptstablemain These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/evolution). In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
Maintenance release with various bug fixes. * GTK UI + config: fix “custom server” setup ([BMC #13511](http://https://bugs.meego.com/show_bug.cgi?id=13511)) When the “default” config template (= ScheduleWorld) was downgraded to “not consumer ready” in SyncEvolution 1.1.0.99.1, setting up a custom SyncML service in the GTK UI stopped working because the UI wouldn’t show the “not consumer ready” config. The problem described above is deterministic and fixed now. Initially the problem seemed to be random. So perhaps there is also another, related issue. * phone sync: delete<->delete conflict + phone calendar+todo sync ([BMC #23744](http://https://bugs.meego.com/show_bug.cgi?id=23744)) When deleting an item on phone and locally, the next sync failed with ERROR messages about “object not found”. Retrying the sync then worked. * Nokia: prevent accidental usage of “calendar” or “todo” sources Nokia phones use a combined “calendar+todo” source for syncing. The “calendar” and “todo” sources also exist because that is where local databases are configured. In such a setup, syncing always has to use “calendar+todo”. For example, to refresh from the Linux desktop to the phone, use: –sync refresh-from-server calendar+todo To work with items (restore, show local content), use the underlying sources, as in: –print-items calendar It was possible to accidentally sync with the “calendar”. This commit prevents that by adding an invalid URI setting to the “calendar” and “todo” sources in the Nokia and Ovi templates. Existing configs are not touched, so beware when you already have configured your Nokia phone. * vCard: X- chat extensions were limited to one instance per kind For example, only one Jabber account could be synchronized. This was caused by an incomplete definition of the conversion to and from vCard. * syncevo-dbus-server + phone sync: catch SIGPIPE to avoid premature exit Frederik Elwert reported that running a local sync with a phone via Bluetooth caused the syncevo-dbus-server to shut down during a sync. Explicitly telling the process to ignore the SIGPIPE signal solved that problem. * syncevo-http-server: support chained SSL certificates So far, the file pointed to by –certificate-file had to contain the server certificated (signed by a CA known to the client) and (optionally) a client certificate. Now the file may also contain additional intermediate certificates which will be sent to the client (chained certificates). * documentation: added glossary and command line conventions sections, improved listing of properties, embedd property definitions in man page, README and README.html * EDS compatibility: fixed inconsistency in libecal check The check for the _r variants in libical still used an older max version. This might have prevented using them (if not found) or could have led to a mixture of old and new libecal in the same process (probably crashed). * glib: avoid including glib/*.h headers directly Recent glib deprecates the direct inclusion of some of its headers, in favor of including glib.h. Doing that here whenever possible, so perhaps it now compiles on Fedora 17 (untested). Upgrading from releases before 1.2 ————————————————– Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Release 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Source, Installation, Further information =============================== Source snapshots are in http://downloads.syncevolution.org/syncevolution/sources i386, lpia and amd64 binaries for Debian-based distributions are available via the “stable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:
debhttp://downloads.syncevolution.org/aptunstablemain These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/evolution). In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
With SyncEvolution 1.2 released and work on 1.3 under way it is a good time to take a small break and reflect on the state of the SyncEvolution project. Features ====== This is meant to give an idea of what can be done with SyncEvolution already. For a more complete list of features, see the documentation about [backends](/documentation/backends), [frontends](/documentation/frontends) and [engine](/documentation/syncevolution-and-synthesis-syncml-engine). This section introduces the key ideas, some of the supported protocols + backends and how they work in combination with specific peers. Data Synchronization ——————————- The main purpose of SyncEvolution is the synchronization of Personal Information Management (PIM) data (contacts, events, tasks, notes). This intentionally does not include email and messages in general because those are sufficiently different to require different solutions. SyncEvolution provides *real* synchronization of PIM data: * Items (= contact, event, …) may have different properties and representations on both sides of a sync (heterogeneous environment). * Items can be modified at any time, even while offline (more than just online access to a server). * Also works between devices, without involving a server. * A database (= a set of items) can be synchronized to one or more peers (other devices, servers). For example, the same address book on a desktop can be synchronized with a mobile phone and a laptop. * SyncEvolution supports arbitrary number of databases. However, many peers (in particular SyncML servers) are more limited and only provide access to one addressbook, calendar, etc. For each peer, SyncEvolution needs to remember the previous state so that the next sync can be limited to only those items which have changed. The underlying assumption is that there is no cycle in the connections between multiple sync peers. For example, “phone <-> laptop, laptop <-> server, phone <-> server” is bad because a new item created on the phone will go to the laptop, from there to the server and then come back as new item to the phone, resulting in a duplicate which then repeats the process ad infinitum. Each SyncEvolution instance only knows the peers it is set up to talk with and thus cannot detect the problem. To break this cycle reliably, each item must have a unique ID that is assigned to it once when the item gets created. For ad-hoc synchronization to work without constraints on the topology, this ID must be supported by all involved peers. Unfortunately, calendar and contact data either doesn’t have such an ID or it is not supported. Without such an ID, each request to add a new item must be checked against all existing items based on some key properties to find out whether it is a duplicate. This is slow and error-prone (“Is ‘Doe, John” the same person as ‘John Doe’, or is one the father and the other a child or namesake?”). *TODO* “[ad-hoc synchronization](https://bugs.meego.com/show_bug.cgi?id=23760) between SyncEvolution instances”: * Teach Evolution Data Server to create and preserve a unique ID for contacts (the vCard 3.0 UID gets overwritten at the moment). * Improve the SyncEvolution<->SyncEvolution sync such that * it uses the unique ID to speed up sync between peers which sync against each other for the first time and * checks the ID to find duplicates in following syncs. *TODO* “[ad-hoc synchronization](https://bugs.meego.com/show_bug.cgi?id=23760) between arbitrary peers”: * Do the duplicate detection based on item content for each add request. Testing ———- SyncEvolution has an extensive test suite which is run regularly (used to be nightly, currently triggered manually). It contains unit tests for various aspects of SyncEvolution and does real interoperability testing with different SyncML, CalDAV and CardDAV servers. See [the SyncEvolution 1.2 test report](/blogs/pohly/2011/syncevolution-12-released#comment-525) for an example. More CalDAV/CardDAV servers were added in the 1.3 branch. Keeping this testing going and analyzing/reporting problems is an on-going activity. It is needed to avoid regressions and achieve higher quality in the releases, which are built as part of these test runs. There are additional ideas. *TODO* “improve nightly testing” * Only the sync code is currently checked under valgrind. Also run [syncevo-dbus-server under valgrind](https://bugs.meego.com/show_bug.cgi?id=5609). * Run [tests in MeeGo (or soon Tizen) chroot](https://bugs.meego.com/show_bug.cgi?id=8684). * Automatically [integrate and test](https://bugs.meego.com/show_bug.cgi?id=735) branches which are considered ready for integration. * Test [syncevo-http-server](https://bugs.meego.com/show_bug.cgi?id=1367). Could do [suspend/resume tests](https://bugs.meego.com/show_bug.cgi?id=1009) against that. SyncML + Engine ————————— The Synthesis engine which provides SyncML is one of the best in the industry. It has extensive support for data modeling/conversion and supports suspending a session and resuming it later. The same engine is also used to synchronize between two backends internally. This is how synchronization was added for CalDAV/CardDAV, protocols which themselves only provide online access. This works reasonably well, but there are also quite a few limitations. *TODO* “[libvxx = refactoring of the Synthesis code base](https://bugs.meego.com/show_bug.cgi?id=23761)” * make data conversion available outside of a sync session * decouple sync engine from SyncML and SyncML message encoding/decoding * support items which are a set of items: important for CalDAV + ActiveSync, because those combine all VEVENTs with the same UID in one item; certain transformations cannot be done in the engine at the moment because they depend on access to all related items at once and that is not how they are handled at the moment * more flexible session handling: instead sending changes in one direction, then back and then stopping, allow the session to continue until both sides are in sync; required for CalDAV where storing an event might lead to further changes that have to be sent back *TODO* “[push sync](https://bugs.meego.com/show_bug.cgi?id=23762)”: * react to local or remote changes immediately (instead of polling at long intervals) and/or * make sync sessions without changes more efficient (in particular when polling) *TODO* “[better credential handling](https://bugs.meego.com/show_bug.cgi?id=23763)” * When creating multiple configurations which need the same credentials (Google CalDAV and SyncML, for example), the username/password needs to be set separately. Replace with a mechanism where both configs only contain a pointer to shared credentials. * Also support an external system component which does the authentication without ever returning username/password to SyncEvolution. Depends on having such a component. This could be used to use a CalDAV/CardDAV server as backend for a SyncML server. *TODO* “[use CalDAV/CardDAV inside SyncML server](https://bugs.meego.com/show_bug.cgi?id=23764)”: * In the backend, check databaseuser/password before falling back to the context’s username/password. * Support shared credentials, to avoid having to configure them in each source. At the moment there are situations where SyncEvolution cannot determine what the right resolution for a failed synchronization is. It has a built-in backup mechanism and can ask the user for assistance, but ultimately it would be better to not bother the user. This can be achieved in some cases by simplifying the problem: * The peer (most likely a service on the Internet) must be able to store all data. * There is a separate local database for each database on the peer. In that case it is acceptable to wipe out the local data and restart with the data stored on the peer. *TODO* “[automatic error recovery](https://bugs.meego.com/show_bug.cgi?id=23765)” * Implement the necessary policy in SyncEvolution (“peer wins”). * Check whether this interferes with libfolks (local IDs will change, data added by libfolks to a contact might get lost). CalDAV/CardDAV ————————– Works reasonable well and passes automated testing against a variety of servers (Apple Calendar Server, DAViCal, Google Calendar, Yahoo). But there are some know limitations, like meeting invitations being sent by Evolution and the CalDAV server. *TODO* “[better support for meeting invitations](https://bugs.meego.com/show_bug.cgi?id=23766)” * Find a way to suppress sending of meeting invitations on the CalDAV server when Evolution already sent one, *or* * suppress sending of meeting invitations in Evolution for calendars which are mirrored in a CalDAV server. * When storing a meeting and/or the server does not return an ETag, retrieve the possibly modified item from the server and store the modified item locally. Right now the more possibly automatically modified data on the server gets ignored. Depends on the engine improvements mentioned above. *TODO* “[handle concurrent changes](https://bugs.meego.com/show_bug.cgi?id=23767)” * Use ETags to avoid modifying more recent data on the server. *TODO* “[CalDAV attachments](https://bugs.meego.com/show_bug.cgi?id=23768)” * At the moment, attachments are not supported at all and even may get lost. Need to preserve them and perhaps even support the more efficient “managed attachments” that are currently being discussed by the CalConnect consortium. *TODO* “[WebDAV: use sync extension](https://bugs.meego.com/show_bug.cgi?id=23769)” * Each sync session must list the entire collection (= retrieve path names and ETag) to determine new and modified items. This can be done more efficiently by using the sync extension defined and implemented in the Apple Calendar Server. *TODO* “[CalDAV tasks = VTODO](https://bugs.meego.com/show_bug.cgi?id=24893) * The CalDAV backend is limited to exchanging VEVENTs. It might also not handle collections well (or at all) that contain both VEVENTs and VTODOs. ActiveSync —————- This was not included in SyncEvolution 1.2 although it was already quite usable. Work on it is still going on. The main issue with ActiveSync is the limited data model specified as part of the protocol. Contacts are only allowed to have a fixed number of certain phone numbers and addresses, which is a limitation that neither Evolution nor Google Contacts have. There is no good way to handle these limitations except educating the user about them, or enforcing the same constraints locally by modifying the app which creates and modifies contacts. The ActiveSync calendar format does not support detached recurrences properly (“you are invited to a specific instance of a meeting series”), although Exchange internally does. *TODO* “finish initial ActiveSync support” * [detached recurrences without parent](https://bugs.meego.com/show_bug.cgi?id=22831): * investigate receiving multiple detached recurrences in multiple items with the same UID in each (the Exchange workaround for the ActiveSync calendar format limitation): might break SyncEvolution * implement the “stand-alone detached recurrence” support (either do it like Exchange does or better, create a fake parent event) * test [concurrent item changes](https://bugs.meego.com/show_bug.cgi?id=23770) while a sync runs (supposed to work, but without a test case it is hard to be sure) **[updated]** *TODO* “ActiveSync performance improvement” * Writing changes to the server is done one change at a time. Could be improved considerably by batching changes. Depends on [core engine improvements](https://bugs.meego.com/show_bug.cgi?id=23967). *TODO* “[ActiveSync push sync](https://bugs.meego.com/show_bug.cgi?id=23771)” * enhance activesyncd and SyncEvolution (see engine above) to react to server-side changes with minimal delay *TODO* “[ActiveSync calendar attachments](https://bugs.meego.com/show_bug.cgi?id=23772)” * At the moment, attachments are not supported at all and even may get lost. Need to preserve them. Not sure how to do it efficiently. Google Calendar ———————— Overall Google Calendar syncing works well with CalDAV (I’m using it myself) as long as one does all meeting scheduling in the Google web interface. But there are some known issues, most of them on the server side: * [stand-alone detached recurrences cannot be accessed, SyncEvolution workaround only works for some cases](http://code.google.com/p/google-caldav-issues/issues/detail?id=58) * [meetings removed by SyncEvolution](http://code.google.com/p/google-caldav-issues/issues/detail?id=59) - not sure yet whether this is better fixed on the server or client side * [cannot updated/delete meetings although the organizer](http://code.google.com/p/google-caldav-issues/issues/detail?id=60): SyncEvolution is allowed to create an event, but then cannot update or remove it * [cannot remove detached recurrence](http://code.google.com/p/google-caldav-issues/issues/detail?id=61): breaks the testLinkedItemsRemoveNormal tests Google Contacts ————————- Works via SyncML. Google’s support for SyncML is very incomplete (many properties not supported, for example birthday). *TODO* “better Google Contacts support” * [Test ActiveSync](https://bugs.meego.com/show_bug.cgi?id=23773) and/or * write [backend based on Google Data Protocol](https://bugs.meego.com/show_bug.cgi?id=23774) - might be the best solution, because ActiveSync also has limitations (see above). Apple Calendar Server, DAViCal ———————————————- Done via CalDAV/CardDAV. No know issues with these peers. Yahoo Calendar + Contacts —————————————– Works well, when it works; unfortunately the number of requests per 24 hour period is so limited that the tests cannot complete without running into a 503 “Service Unavailable” error. *TODO* “[Yahoo token authentication](https://bugs.meego.com/show_bug.cgi?id=23775)” * Currently SyncEvolutions uses normal HTTP authentication. Yahoo also supports another, token based authentication mechanism for approved apps. Get SyncEvolution approved and implement that other authentication mechanism. Hopefully that’ll avoid the 503 error. Phones ———– SyncEvolution can synchronize against phones if those phones support SyncML via Bluetooth. This is a common feature among older feature phones but most (all?) Android phones and iPhones don’t support it. Testing of this feature is limited and there is no list of phones which are guaranteed to work. **[updated]** Synchronization with Android/iOS is possible by installing a third-party SyncML client (like the ones from Synthesis) and configuring the phones to use a [SyncEvolution HTTP SyncML server](/wiki/synchronizing-evolution-http-howto). *TODO* “[contact sync via PBAP](https://bugs.meego.com/show_bug.cgi?id=23776)” * Write a backend using PBAP, the only (?) protocol supported by Android and iOS for address book access. Not very good for real syncing, but at least one-way sync should be possible. GTK UI ———– The GTK “sync UI” is the main user interface for SyncEvolution in MeeGo and Linux in general. It is included in the source and binary distribution archives. It supports configuring syncing against SyncML services (extensible via configuration templates) and phones which support SyncML. Configuring the latter is integrated into the GNOME Bluetooth applet, which invokes the sync UI. The UI also has “emergency recovery” support which allows the user to restore from the automatic backup and/or choose between different recovery operations after a failed sync: * continue with local data * continue with remote data * try to merge both (“slow sync”) *TODO* “[port to GTK3](https://bugs.meego.com/show_bug.cgi?id=23094)” * The sync UI is currently using GTK2. It needs to be ported to GTK3. The goal is to put the GTK2 version into maintenance and continue with the GTK3 version. *TODO* “[configure local sync, choose databases](https://bugs.meego.com/show_bug.cgi?id=23777)” * CalDAV/CardDAV and ActiveSync cannot be configured yet with the UI. In this context it becomes important to let the user choose local and remote databases. Would be useful to have, although there is a certain overlap with the Evolution integration. Evolution ————- This happens to be the main backend for storing data locally. Other backends could be supported just as well (there’s nothing technical which favors Evolution) if there were developers motivated enough to implement and test them. KDE/Akonadi has come a long way, but seems less active (see KDE/Akonadi and Community below). *TODO* “Evolution backend improvements” * use new APIs in EDS 3.2/3.4 for more [efficient change detection](https://bugs.meego.com/show_bug.cgi?id=23778) * enable [creating databases](https://bugs.meego.com/show_bug.cgi?id=23779) from inside SyncEvolution again (depends on EDS 3.4 (?)) *TODO* “[integrate SyncEvolution into Evolution](https://bugs.meego.com/show_bug.cgi?id=23780)” * Evolution typically supports offline read access with some of its backends. It does not support write access. SyncEvolution addresses that, but needs to be configured and invoked separately. It would be nice to have seamless and transparent syncing from inside Evolution. ActiveSync is getting integrated like that at the moment -> extend that to CalDAV/CardDAV? File backend —————— There is a generic file backend with a 1:1 mapping between a single item and a file in a directory. The format of the local data can be configured. vCalendar 1.0, iCalendar 2.0, vCard 2.1/3.0 and plain text notes are supported already. *TODO* “[iCalendar 2.0 .ics file](https://bugs.meego.com/show_bug.cgi?id=23781)” * Some people synchronize an .ics file by pointing the Evolution backend to it (no longer works after EDS removed support for the file:// URI) or manipulating the ~/.evolution/calendar data (a hack which only works when being very careful). Write a SyncEvolution backend which reads/writes an iCalendar 2.0 file using libical. KDE/Akonadi ——————– The Akonadi backend is available in the source code. Support for KWallet instead of GNOME keyring is also there. A KDE GUI is in development. *TODO* “[test and release binaries](https://bugs.meego.com/show_bug.cgi?id=23782)” * nightly builds needs to be reconfigured to enable building the KDE support * nightly testing needs to include KDE backends, both for unit testing the backends and real combinations with peers; it depends on the KDE developers to do something if issues are found in those tests (if any are found) Manipulating PIM data ——————————– SyncEvolution also is a very capable tool for manipulating databases via the command line. It can list, import/export and delete items. Converting between formats (for example, mirror Evolution contacts as files in vCard 2.1 format) is possible by setting up synchronization with a file backend as peer. *TODO* “improve item manipulation” * the command line syntax is a bit awkward and depends on dummy configurations: [make config+source names optional](https://bugs.meego.com/show_bug.cgi?id=23783) * [convert between formats](https://bugs.meego.com/show_bug.cgi?id=23784) (depends on libvxx engine improvements) Community ======== Lots of users. The feedback from users is often very helpful for improving the software. Not so many developers, though, except those paid to work on SyncEvolution. Ove Kaaven (port+UI for Maemo/N900 and MeeGo Harmattan/N9/N950) and Frederik Elwert (Genesis UI) are the notable exceptions. Sascha Peilicke/Dinesh/Rohan Garg have done some work for KDE/Akonadi, but are not very active in the SyncEvolution project itself. I can only speculate about the reasons for the lack of external contributors: * The number of people who care about PIM sync and storage is small to start with. It’s not one of those sexy areas that gets a lot of attention, although it is arguably very important. * OpenSync still seems to be considered “the” open source sync solution, despite not having a stable release available or even in sight anytime soon. * Perhaps developers also get the impression that problems will be solved anyway, without having to get involved. That is true for some aspects, but definitely not for all. * SyncEvolution development has a certain learning curve (although Ove and Franz Knipp managed to write their backends with very little assistance) and often happens at a rapid pace, which makes it hard for people to contribute small improvements. * SyncEvolution was focused on Evolution and SyncML initially, which might make it seem too limited in scope for some use cases. But the scope has already increased and will increase further, so now would be a good time to check it out again. Many of the TODOs above will not get worked on unless some external developer picks up the mantle and contributes patches. I’d be more than happy to help someone get started. If there is interest, I can also tag bugs in Bugzilla as “easy fixes”. Right now I am not doing that because it would create additional work which would not be justified if there is no-one interested - I know, chicken and egg…
The major new feature of the 1.2 release is support for non-SyncML protocols in general and CalDAV/CardDAV in particular. ActiveSync support is in development and will be in 1.3. These protocols are implemented as backends which are combined with other backends by SyncEvolution in a so called “local sync”. The GTK sync-ui does not yet support configuring non-SyncML protocols. See the [README.rst and man page](/documentation/syncevolution-usage) for more information on how to use the new feature via the command line. Properties not supported by SyncML servers can now be preserved locally in two-way synchronization ([BMC #15030](https://bugs.meego.com/show_bug.cgi?id=15030)). This depends on information about what properties a SyncML server supports (“CtCap”), which is typically not provided by servers. SyncEvolution contains a copy of that information for Google Contacts ([BMC #15029](https://bugs.meego.com/show_bug.cgi?id=15029)). Akonadi backend and KWallet support were merged. They are not included yet in syncevolution.org binaries. To use them compile from source. The configuration format was updated to solve a conceptual problem inherited with the legacy property names: the “type” property had multiple, sometimes conflicting roles. For example, setting the preferred data format for sync with one peer might have changed the backend selection for some other peer ([BMC #1023](https://bugs.meego.com/show_bug.cgi?id=1023)). Now “backend/databaseFormat/syncFormat/forceSyncFormat” replace “type”. “type” is still accepted by the command line as alias. Upgrading from releases before 1.2 ———————————– Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Release 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Other changes ————- * Using the –sync-property and –source-property command line options is optional, just specifying the property assignment is enough. * syncevo-http-server was enhanced considerably. See http://syncevolution.org/wiki/http-server-howto * support NetworkManager API >= 0.9 ([BMC #19470](https://bugs.meego.com/show_bug.cgi?id=19470)) * syncevolution.org binaries: now compatible with Debian Testing/libnotify.so.4 ([BMC #22668](https://bugs.meego.com/show_bug.cgi?id=22668)) libnotify is not linked directly into syncevo-dbus-server in the syncevolution.org binaries. Instead libnotify.so.1 till .so.4 (current Debian Testing) are opened opened dynamically and the necessary functions are looked up via dlsym(). Not finding the libraries or the functions silently disables this notification mechanism. * Sync mode is recorded when running in SyncML server mode ([BMC #2786](https://bugs.meego.com/show_bug.cgi?id=2786)). * syncevo-dbus-server automatically stops when some of its libraries are updated and restarts if auto-syncing is on ([BMC #14955](https://bugs.meego.com/show_bug.cgi?id=14955)). * Added code for Buteo, mKCal and QtContacts in MeeGo. Buteo and mKCal were removed again from MeeGo, so the code is obsolete. The QtContacts backend may be still be useful to access items via that API, but for syncing on MeeGo the normal EDS backend is used since MeeGo reverted back to EDS as PIM storage. * “databasePassword” source property: lookup failure in keyring ([BMC #22937](https://bugs.meego.com/show_bug.cgi?id=22937)) The databasePassword also wasn’t looked up at all when doing item operations via the command line. When configuring sources for an HTTP server, the config name typically is just the context (@foo). When using the config in the HTTP server, the config name is the peer inside that context (client@foo). Because the GNOME keyring lookup keys for the “databasePassword” (more specifically, the object name) contained the full config name which was different in both cases, looking up the saved password failed. The solution is to normalize the config name (to accomodate for different ways of spelling it) and use only the context, with @ as before. This will break existing setups where the object name in the keyring (incorrectly) includes the full config name. In that case just configure the source again to set the password anew. * Evolution Calendar: fixed detached recurrence support ([BMC #22940](https://bugs.meego.com/show_bug.cgi?id=22940)) When manipulating a meeting series with more than one detached recurrence certain sequences of operations could incorrectly fail with “UID already exists”. * iCalendar 2.0: must set VALUE in EXDATE (part of [BMC #22940](https://bugs.meego.com/show_bug.cgi?id=22940)) EXDATE has a VALUE parameter, which wasn’t defined in the XML profile. Didn’t seem to matter at all in practice, but wasn’t standard-compliant. * GTK sync-ui: wrap sync service descriptions ([BMC #7199](https://bugs.meego.com/show_bug.cgi?id=7199)) Descriptions of different sync services are not fully visible unless word-wrapping gets enabled. * CalDAV/CardDAV + local storage: avoid empty properties The main motivation for this change is that a recent Apple Calendar server rejects vCards with empty BDAY property. Another reason is that keeping the data as small as possible is desirable by itself. Sending an empty property serves as a hint for the peer that the property is supported. This is not necessary when storing an item in a backend. Therefore this commit disables empty properties for all backends which do not themselves set the m_backendRule Synthesis info value. * Google Contacts: ensure that first/middle/name are set when storing in EDS ([BMC #20864](https://bugs.meego.com/show_bug.cgi?id=20864)) Evolution and the MeeGo UX assume that first/middle/last name are set. That is not the case when a contact is created in the Google Contacts web interface. Such contacts are sent by Google without the N property. SyncEvolution now tries to recreate the name components from the FN string, by splitting at word boundaries and assuming “” or “, ” format. Obviously this heuristic fails for some locales. * Evolution Calendar: fixed error handling for broken TZIDs * Sony Ericsson: use ISO-8859-1 for all devices ([BMC #14414](https://bugs.meego.com/show_bug.cgi?id=14414)) Passing invalid UTF-8 strings into libecal caused glib to abort syncevo-dbus-server. * auto sync: show all failed syncs except for temporary network errors ([BMC #21888](https://bugs.meego.com/show_bug.cgi?id=21888)) Notifications were meant to be shown for all errors except temporary ones. This has never been implemented correctly since the feature was introduced: instead of hiding known temporary errors, all errors except 500 (fatal error) were suppressed. * vCard: inline local photo data ([BMC #19661](https://bugs.meego.com/show_bug.cgi?id=19661)) Some platforms (Maemo, MeeGo) store photos in separate files. Now SyncEvolution efficiently includes that photo data in the generated vCard right before sending it to a peer; previously it sent a useless local file:// URI. The Maemo port has a less efficient workaround for that which now should be obsolete. * syncevo-dbus-server: online status wrong without Network Manager or ConnMan ([BMC #21543](https://bugs.meego.com/show_bug.cgi?id=21543)) When neither Network Manager nor ConnMan are running, network presence was “not online”. This prevented running automatic syncs. For developers: * modified backend API - ClientTestConfig modernized - InsertItemResult::m_merged turned from boolean to enum * testing and compilation changes; for example, the minimum version of libsynthesis is now checked at configure time instead of failing at runtime due to missing features in the Synthesis engine SyncEvolution 1.1.99.7 -> 1.2, 13.10.2011 ========================================= Some more bug fixes and testing improvements. * fixed potential invalid memory access in add<->add conflict handling * fixed memory leak in workaround for EDS bug * CalDAV/CardDAV: handle ETags without quotation marks (eGroupware) * updated README: warning about sync direction moved to –sync option Source, Installation, Further information =============================== Source snapshots are in http://downloads.syncevolution.org/syncevolution/sources i386, lpia and amd64 binaries for Debian-based distributions are available via the “stable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:
debhttp://downloads.syncevolution.org/aptstablemain These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/evolution). In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
Krzesimir Nowak [wrote about his work](http://krnowak.blogspot.com/2011/10/syncevolution-build-system-work.html) on converting SyncEvolution from an autotools project with recursive make to non-recursive make. Definitely worth a read for anyone interested in autotools. The current SyncEvolution master branch (post 1.2) uses that new build system. His conclusion is that the new system is not necessarily easier to understand than the one before (autotools with some preprocessing shell scripts). Partly that’s because SyncEvolution tries to achieve certain things not supported well by autotools (automatically generated version number, avoid listing all files explicitly, backends which can add to the global configure script), partly it is because non-recursive Automake introduces additional constraints (like having to avoid clobbering variables and rules). Helper scripts are still needed, the only difference is how they get called. Either way, what sold me on the idea of a non-recursive make is that on a machine with many cores, like the SyncEvolution nightly test server, compilation is considerably faster because parallel make can spawn more jobs in parallel. Recursive make often had to wait for the completion of compilation in a sub-directory. If memory serves me right, it was more than twice as fast.
Mostly bug fixes again. Some are a bit more intrusive, thus another pre-release. Changes ======= * syncevolution.org binaries: now compatible with Debian Testing/libnotify.so.4 ([BMC #22668](https://bugs.meego.com/show_bug.cgi?id=22668)) libnotify is not linked directly into syncevo-dbus-server in the syncevolution.org binaries. Instead libnotify.so.1 till .so.4 (current Debian Testing) are opened opened dynamically and the necessary functions are looked up via dlsym(). Not finding the libraries or the functions silently disables this notification mechanism. * calendar sync: better handling for add<->add conflicts (partly fixes [BMC #22783](https://bugs.meego.com/show_bug.cgi?id=22783)) When both sides of a sync have added the same event, the sync must determine which one is more recent instead of blindly overwriting always the same side. Such conflicts are typically rare except for enterprise scenarios where meeting invitiations are processed automatically by a groupware (Exchange, Google Calendar/Mail, …) and then the attendee status is updated on one side. SyncEvolution now does the necessary age comparison and preserves the more recent data for most properties. In some properties the data from both sides is preserved by concatenating the text (description, location, …). It remains to be seen whether that is really desirable. Also, sync statistics are slightly off: the incoming item is counted as “added” even though it gets turned into an update. * item operations: authentication problem for WebDAV when using keyring ([BMC #21311](https://bugs.meego.com/show_bug.cgi?id=21311)) The password still wasn’t looked up in the keyring when using –import/export/delete-items. * “databasePassword” source property: lookup failure in keyring ([BMC #22937](https://bugs.meego.com/show_bug.cgi?id=22937)) The databasePassword also wasn’t looked up at all when doing item operations via the command line. When configuring sources for an HTTP server, the config name typically is just the context (@foo). When using the config in the HTTP server, the config name is the peer inside that context (client@foo). Because the GNOME keyring lookup keys for the “databasePassword” (more specifically, the object name) contained the full config name which was different in both cases, looking up the saved password failed. The solution is to normalize the config name (to accomodate for different ways of spelling it) and use only the context, with @ as before. This will break existing setups where the object name in the keyring (incorrectly) includes the full config name. In that case just configure the source again to set the password anew. * Evolution Calendar: fixed detached recurrence support ([BMC #22940](https://bugs.meego.com/show_bug.cgi?id=22940)) When manipulating a meeting series with more than one detached recurrence certain sequences of operations could incorrectly fail with “UID already exists”. * iCalendar 2.0: must set VALUE in EXDATE (part of [BMC #22940](https://bugs.meego.com/show_bug.cgi?id=22940)) EXDATE has a VALUE parameter, which wasn’t defined in the XML profile. Didn’t seem to matter at all in practice, but wasn’t standard-compliant. * GTK sync-ui: wrap sync service descriptions ([BMC #7199](https://bugs.meego.com/show_bug.cgi?id=7199)) Descriptions of different sync services are not fully visible unless word-wrapping gets enabled. * source configs: don’t check “backend” unless it is needed When using a config which has sources with a backend type set which is not currently available, an error was thrown even if those sources weren’t even part of the current operation (for example, syncing another source which is currently supported). * config migration: avoid name conflicts and auto syncing of old configs ([BMC #22691](https://bugs.meego.com/show_bug.cgi?id=22691)) When (auto-)migrating a config, it was possible that a name for the peer, say foo.old, was chosen for the renamed config although there was already such a config, for example foo.old in ~/.sync4j. Besides being confusing for users, this also led to a bug in the code where it copied from the older config with the foo.old name. The main problem fixed is the disabling of auto syncing in the old config. Otherwise it was still used by syncevo-dbus-server for syncing, which triggered another auto-migration, ad infinitum… * auto syncing: must check whether enabled when looking at unknown URLs (part of [BMC #22691](https://bugs.meego.com/show_bug.cgi?id=22691)) “syncURL = insert your URL here” with “autoSync = 0” did lead to auto sync attempts although it wasn’t enabled. A check for “auto syncing enabled” was missing for the “unknown transport” case. * CalDAV/CardDAV + local storage: avoid empty properties The main motivation for this change is that a recent Apple Calendar server rejects vCards with empty BDAY property. Another reason is that keeping the data as small as possible is desirable by itself. Sending an empty property serves as a hint for the peer that the property is supported. This is not necessary when storing an item in a backend. Therefore this commit disables empty properties for all backends which do not themselves set the m_backendRule Synthesis info value. * Apple CardDAV: apply PHOTO import/export scripts by default A recent Apple Calendar server (correctly) rejects the invalid PHOTO;TYPE=unknown: property in a vCard. This internal representation must be cleared before serializing the field list. * for developers: modified backend API - ClientTestConfig modernized - InsertItemResult::m_merged turned from boolean to enum * testing and compilation changes; for example, the minimum version of libsynthesis is now checked at configure time instead of failing at runtime due to missing features in the Synthesis engine Source, Installation, Further information =============================== Source snapshots are in http://downloads.syncevolution.org/syncevolution/sources i386 and amd64 binaries for Debian-based distributions are available via the “unstable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:
debhttp://downloads.syncevolution.org/aptunstablemain Binaries for lpia will be made available again in the next release. These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in http://downloads.syncevolution.org/syncevolution/evolution. In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
Mostly bug fixes, some improvements in testing and packaging. This release was tested successfully with DAViCal 0.9.9.4. Changes ======= * CalDAV: fixed incorrect change tracking causing “event not found” ([BMC #22329](https://bugs.meego.com/show_bug.cgi?id=22329)) * CalDAV: handle delete<->delete conflict during local sync ([BMC #22327](https://bugs.meego.com/show_bug.cgi?id=22327)) If the same event was deleted both locally and in the CalDAV server, syncing failed with “event not found”. * Google Contacts: ensure that first/middle/name are set when storing in EDS ([BMC #20864](https://bugs.meego.com/show_bug.cgi?id=20864)) Evolution and the MeeGo UX assume that first/middle/last name are set. That is not the case when a contact is created in the Google Contacts web interface. Such contacts are sent by Google without the N property. SyncEvolution now tries to recreate the name components from the FN string, by splitting at word boundaries and assuming “” or “, ” format. Obviously this heuristic fails for some locales. * CalDAV: continue despite Google Calendar access problems (see [BMC #19484](https://bugs.meego.com/show_bug.cgi?id=19484)) An attempt to work around “403 You don’t have access to change that event” errors, perhaps caused by http://code.google.com/p/google-caldav-issues/issues/detail?id=38 The problem is now recorded instead of aborting the sync. The sync then ends in a 22001 = “partial failure” error and the operation will be retried in the next sync. * CalDAV: transform UTC RECURRENCE-ID for Evolution ([BMC #22594](https://bugs.meego.com/show_bug.cgi?id=22594)) Evolution showed a meeting twice on the day of a modified recurrence, if the meeting series was originally created and modified in Exchange, then imported into Google Calendar. * CalDAV syncevolution.org binaries now works when libneon.so.27 or libneon-gnutls.so.27 (Debian) are installed. Previously libneon.so.27 was required, which is no longer available in Debian Testing. * syncevo-dbus-server/gdbus: fixed segfault when asked for properties when none are available ([BMC #22152](https://bugs.meego.com/show_bug.cgi?id=22152)) * Evolution Calendar: fixed error handling for broken TZIDs * Sony Ericsson: use ISO-8859-1 for all devices ([BMC #14414](https://bugs.meego.com/show_bug.cgi?id=14414)) Passing invalid UTF-8 strings into libecal caused glib to abort syncevo-dbus-server. * item operations: authentication problem for WebDAV when using keyring ([BMC #21311](https://bugs.meego.com/show_bug.cgi?id=21311)) The password wasn’t looked up in the keyring when using –print-items/import/export/… * WebDAV: fixed item operations without configuration ([BMC #22164](https://bugs.meego.com/show_bug.cgi?id=22164)) Previously failed with “[ERROR] : virtual read-only configuration node, cannot write property webDAVCredentialsOkay = 1”. * auto sync: show all failed syncs except for temporary network errors ([BMC #21888](https://bugs.meego.com/show_bug.cgi?id=21888)) Notifications were meant to be shown for all errors except temporary ones. This has never been implemented correctly since the feature was introduced: instead of hiding known temporary errors, all errors except 500 (fatal error) were suppressed. * vCard: inline local photo data ([BMC #19661](https://bugs.meego.com/show_bug.cgi?id=19661)) Some platforms (Maemo, MeeGo) store photos in separate files. Now SyncEvolution efficiently includes that photo data in the generated vCard right before sending it to a peer; previously it sent a useless local file:// URI. The Maemo port has a less efficient workaround for that which now should be obsolete. * syncevo-dbus-server: online status wrong without Network Manager or ConnMan ([BMC #21543](https://bugs.meego.com/show_bug.cgi?id=21543)) When neither Network Manager nor ConnMan are running, network presence was “not online”. This prevented running automatic syncs. * fixed compile issues with Debian Testing/gcc 4.6.1 Known issues, might still be resolved for the final 1.2 ====================================== * syncevolution.org binaries: libnotify1 -> libnotify4 incompatibility ([BMC #22668](https://bugs.meego.com/show_bug.cgi?id=22668)) Newer distros no longer have the libnotify.so.1 that syncevolution.org binaries depend on. As a workaround it is possible to install the libnotify1 package from older distro releases. * CalDAV: add<->add conflicts ([BMC #22669](https://bugs.meego.com/show_bug.cgi?id=22669)) Suppose the same meeting invitation for event UID=FOO is processed in both Evolution and Google Calendar. This always happens when the meeting invitation emails is sent to Google Mail, then later viewed in Evolution. On the Evolution side, the invitation is accepted. In Google Calendar this is still open. When syncing in that state the sync engine does not recognize that both sides have added the same meeting and the “meeting accepted” information eventually gets lost. As a workaround, always synchronize the calendar before processing meeting invitation emails. Source, Installation, Further information =============================== Source snapshots are in http://downloads.syncevolution.org/syncevolution/sources i386 and amd64 binaries for Debian-based distributions are available via the “unstable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:
debhttp://downloads.syncevolution.org/aptunstablemain Binaries for lpia will be made available again in the next release. These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in http://downloads.syncevolution.org/syncevolution/evolution. In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
Release 1.1.99.5 is the first release candidate for 1.2. It has gone through a long stabilization period and thus is suitable for normal users. The major new feature of the 1.2 release is support for non-SyncML protocols in general and CalDAV/CardDAV in particular. ActiveSync support is in development. Changes 1.1.1 -> 1.1.99.5 ==================== The major new feature of the 1.2 release is support for non-SyncML protocols in general and CalDAV/CardDAV in particular. ActiveSync support is in development. These protocols are implemented as backends which are combined with other backends by SyncEvolution in a so called “local sync”. The GTK sync-ui does not yet support configuring non-SyncML protocols. See the [README and man page](/documentation/syncevolution-usage) for more information on how to use the new feature via the command line. Properties not supported by SyncML servers can now be preserved locally in two-way synchronization ([BMC #15030](http://bugs.meego.com/show_bug.cgi?id=15030)). This depends on information about what properties a SyncML server supports (“CtCap”), which is typically not provided by servers. SyncEvolution contains a copy of that information for Google Contacts ([BMC #15029](http://bugs.meego.com/show_bug.cgi?id=15029)). Akonadi backend and KWallet support were merged. They are not included yet in syncevolution.org binaries. To use them compile from source. The configuration format was updated to solve a conceptual problem inherited with the legacy property names: the “type” property had multiple, sometimes conflicting roles. For example, setting the preferred data format for sync with one peer might have changed the backend selection for some other peer ([BMC #1023](http://bugs.meego.com/show_bug.cgi?id=1023)). Now “backend/databaseFormat/syncFormat/forceSyncFormat” replace “type”. “type” is still accepted by the command line as alias. Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. In contrast to earlier, more experimental releases in the 1.2 series, 1.1.99.5 and later automatically migrate configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Other changes: * a problem with enabling release mode required replacing 1.1.99.5 with a fixed 1.1.99.5a release * syncevo-http-server was enhanced considerably. See the [HTTP server HOWTO](/wiki/http-server-howto) * support NetworkManager API >= 0.9 ([BMC #19470](http://bugs.meego.com/show_bug.cgi?id=19470)) * Sync mode is recorded when running in SyncML server mode ([BMC #2786](http://bugs.meego.com/show_bug.cgi?id=2786)). * syncevo-dbus-server automatically stops when some of its libraries are updated and restarts if auto-syncing is on ([BMC #14955](http://bugs.meego.com/show_bug.cgi?id=14955)). * Using the –sync-property and –source-property command line options is optional, just specifying the property assignment is enough. * Added support for Buteo, mKCal and QtContacts in MeeGo. Buteo and mKCal were removed again from MeeGo, so the code is obsolete. The QtContacts backend may be still be useful to access items via that API, but for syncing on MeeGo the normal EDS backend is used since MeeGo reverted back to EDS as PIM storage. * code cleanup and various minor fixes/improvements, see ChangeLog Source, Installation, Further information =============================== Source snapshots are in http://downloads.syncevolution.org/syncevolution/sources i386, amd64 and lpia binaries for Debian-based distributions are available via the “unstable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:
debhttp://downloads.syncevolution.org/aptunstablemain These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in http://downloads.syncevolution.org/syncevolution/evolution. In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
David Bremner, the Debian packager of SyncEvolution, [announced](http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=624540) that he is looking for a new maintainer to take over that package. If you care about SyncEvolution in Debian and Ubuntu, then please consider taking over. On this occasion let me thank David Bremner for getting SyncEvolution into Debian and for maintaining it there.
There have been two blog posts recently who point out that data synchronization using open source tools still doesn’t work as well as it should: * [Adam Williamson: “The continuing state of contact + calendar synchronization suck”](http://www.happyassassin.net/2011/04/13/the-continuing-state-of-contact-calendar-synchronization-suck) * [Matěj Cepl: Synchronization sucks!](http://luther.ceplovi.cz/blog/2011/04/synchronization-sucks/) As someone who has worked hard on making synchronization suck less I can’t let this stand without commenting… in particular not when SyncEvolution is mentioned as not being reliable! ;-) Nitpicking ======= First let me point out some factual mistakes in Adam’s post: *”Maemo’s whole synchronization story”* has never been based on SyncEvolution. SyncEvolution is an add-on, supported entirely on a volunteer basis by Ove Kaveh on the device. If people do not get help for SyncEvolution in the Maemo forums, then I can’t do anything about it because I don’t have the time to keep up with everything that is said (or asked) on the web about SyncEvolution. It’s up to the Maemo community to help their peers. SyncEvolution on Linux to N900 [broken somewhere outside of SyncEvolution](https://bugs.meego.com/show_bug.cgi?id=4835) (device, Bluetooth stack) and without support by Nokia for their closed-source sync component on the device I don’t see much chances to fix it, short of some Bluetooth experts getting involved. The statements about MeeGo are also incorrect. Evolution Data Server was not the preferred PIM storage for MeeGo 1.2 until recently, so depending on it for CalDAV/CardDAV support was not an option. Adam points to [a bug report](https://bugs.meego.com/show_bug.cgi?id=319) where I captured some thoughts around the technical aspects. Perhaps this was too brief to be understand without context, but I still think that the arguments and conclusion are valid. More on that below. What exactly is the complaint? ======================= Adam tried SyncEvolution with Horde and eGroupware. Other people were able to get these combinations working, for example just a few days ago [George Runelli with eGroupware](http://www.ruinelli.ch/how-to-sync-egroupware-with-a-tablet-n900-with-syncevolution). It seems to depend a lot on the exact setup on the server side. I had offered Adam to help with diagnosing his problem with Horde (unexpected slow syncs), but he never replied to my email. I’m still convinced that the problem is not in SyncEvolution, but rather on the server side, because SyncEvolution works fine with a variety of other SyncML servers (Funambol, Memotoo, Synthesis, Mobical, to name just those that I test with nightly). It is not SyncEvolution’s fault that the open source groupwares seem to have less stable SyncML support. I tried to work with Horde and eGroupware developers a while back when I started with SyncEvolution. I had a hard time getting anyone to reply to my questions and emails, even when contacting the original developers directly. If the situation is different today, then I’d be happy to restart that effort. I’m not sure what kind of problem Matěj had with SyncEvolution. He doesn’t say in his blog post, only that it does not allow him to reliably sync with his server running Zarafa. I’m not surprised. To the best of my knowledge, the two are unable to synchronize against each other by design, because SyncEvolution is based on SyncML and Zarafa on ActiveSync. So is the complaint that SyncEvolution uses an open protocol and not a patent-encumbered proprietary protocol? Proposed solutions ============== Adam then continues to suggest that the data synchronization model itself is flawed and should be replaced with client/server model where changes are always stored on the server immediately, as in Evolution’s CalDAV and CardDAV backends. This became more clear in an email discussion after he contacted me regarding his blog post. The key difference is this: * True synchronization allows offline modification of the data. * Capable devices by design store a complete copy of the data, without depending on one particular server to remain online. Adam argued that a client/server model can be combined with caching of items and changes. But then the client/server model **becomes** synchronization and must deal with the same kind of problems that it was meant to avoid, like conflicts between items on client and server. Adam [later said](http://luther.ceplovi.cz/blog/2011/04/synchronization-sucks/#comment-37) that changes that cannot be stored anymore should simply be discarded. That doesn’t sound like a very useful approach, because users won’t be able to remember what changes might have been lost and if they do, would most likely be forced to redo them manually. PIM data is more complex then plain text, so the merge strategies that programers know how to use with source code and revision control systems do not apply. Normal users will be even more challenged. The point about not depending on a central server is important, too. In the SyncML world we have recently seen that ScheduleWorld shut down. No data was lost, because by design all users always had a full copy of their data on their own storage. The same can’t be said for all the popular Web 2.0 cloud services… My devices are not always online, for practical and economical reasons. Therefore I want the ability to make changes while offline and will continue to work on the more capable model. SyncML ====== One other aspect is the question whether the data synchronization model itself is flawed, just some protocols implementing it, or only specific implementations of these protocols. I think the approach itself is sound and useful. Adam’s own observation that other implementations of the concept seem to work better confirms that. But SyncML definitely has its flaws, both in the protocol itself and in implementations. SyncML tries to be too flexible for its own good. It allows the implementation of very dumb devices. The downside is an increased complexity on the server side. Because of its open nature, there have been a variety of implementations with varying degrees of capabilities both in the data that is supported and in the quality of the protocol implementation itself. That makes it challenging today to support the whole range of SyncML capable peers. In that sense, SyncML is a victim of its own success. The silver lining ================= I have some hopes today for CalDAV/CardDAV based synchronization. SyncEvolution 1.2 will have support for that, natively. A native implementation has the conceptual advantage that it can use meta data (resource URI + eTag) to speed up change detection, something that wouldn’t be possible when going through Evolution Data Server. CalDAV enforces that each item must have a globally unique ID, which is a considerable simplification for implementing synchronization. CardDAV unfortunately still doesn’t. Good open source implementations exist, for example Apple’s Calendar server. It passes all of the automated SyncEvolution tests. I also hear good things about DAViCal; unfortunately I haven’t found the time to test with it yet. What might be missing in both cases is good integration into a groupware solution, for those who need that.
Quite a bit of spam accumulated in the site comments. Usually I deleted spam comments within a few days after receiving the comment notification email, but not all spam comments triggered those, so I missed quite a lot. I went back and reviewed all comments and deleted as necessary, so the site should be cleaner now. If I missed anything, please let me know. In order to combat spam, all comments are now held for review before being posted. This will lead to delays, so we plan to improve the anti-spam measures and remove the need to review comments manually again. If someone wants to help maintain the site, such help would be highly appreciated! It takes away time that could be spent on improving the software instead…
A Sony Ericsson K800 user reported a problem with non-standard characters (German umlauts in his case). It turned out that the phone uses ISO-8859-1 instead UTF-8 as encoding, without announcing that. It is possible to configure the Synthesis engine so that it does with the character set conversion correctly, but that leads to this question: *which phones need this special treatment*? I’m currently inclined to enable this for all Sony Ericsson phones, unconditionally. If you are using a Sony Ericsson a) which uses something else than ISO-8859-1 as local charset and/or b) which works correctly with non-standard characters already in SyncEvolution 1.1.1, then please leave a comment.
Maintenance release, in particular improving syncing with phones. There was a bug that could cause all kinds of weird behavior after a failed sync with a phone, so updating is highly recommended. Changes 1.1 -> 1.1.1 ==================== * Synthesis engine: fixed a corruption issue in internal meta data which caused duplicates and other problems in a pretty indeterminstic way; apparently caused by failed syncs ([BMC #11044](http://bugs.meego.com/show_bug.cgi?id=11044)). * Synthesis engine: recurrence rules with end date now sent correctly to phones ([BMC #11241](http://bugs.meego.com/show_bug.cgi?id=11241)). The RRULE property was not encoded correctly previously during the iCalendar 2.0 -> vCalendar 1.0 conversion. Events with recurrence count were okay. Probably also affected SyncML servers without iCalendar 2.0 support. The fix was confirmed to work with Nokia phones. It also helps with Sony Ericsson phones, but at least the t700 still has a problem: depending on the phone’s time zone, it repeats the event for one day too long ([BMC #10092](http://bugs.meego.com/show_bug.cgi?id=10092)). * Synthesis engine: fixed broken time zone information when sending to phone; previously that broke sending calendar updates to Nokia phones ([BMC #9600](http://bugs.meego.com/show_bug.cgi?id=9600)). iCalendar 2.0 time zone definitions imported from libical were not encoded correctly in vCalendar 1.0 items as sent to phones. Nokia phones accepted such data when part of a new event, but rejected updates of it. * Synthesis engine: shorter TZIDs, might help N900 calendar ([BMC #6680](http://bugs.meego.com/show_bug.cgi?id=6680)). The shorter TZIDs will be included in iCalendar 2.0 data exported by libsyntesis and thus SyncEvolution. This change is motivated primarily by the observation that the N900 calendar storage can handle TZID=, but not TZID=/softwarestudio.org/Tzfile/. * ScheduleWorld: disable configuration template because service has shut down. The template is only hidden from the GTK sync-ui, but remains in SyncEvolution for the time being because it is referenced in several places. * Evolution CalDAV: added workaround for “must sync twice” ([BMC #10265](http://bugs.meego.com/show_bug.cgi?id=10265)) The Evolution CalDAV backend seems to update its data when closing the database, not when opening it. As a result, syncevolution had to be run twice to see all data changes. The workaround is to open the database twice at the start of the sync. This is done for all calendar databases, regardless of which backend they use, in case that some other (yet unknown) backend needs the same workaround. * GTK sync-ui: workaround for “Sync Now” button not reacting to online status changed ([BMC #9949](http://bugs.meego.com/show_bug.cgi?id=9949)). * Changed slow sync handling. Some users have complained about getting duplicated contacts ([BMC #10081](http://bugs.meego.com/show_bug.cgi?id=10081)). The exact reason is not known (no useful logs provided yet), but it might be due to using “duplicate” as resolution strategy during slow syncs. This caused slightly different contacts to be duplicated instead of merging the two copies, reasoning that “no data loss” is better than “duplicates”. This release switches to a mode where the engine tries harder to avoid duplicates by merging data if modification time stamps are available for contacts (usually they are). When fields differ, the more recent data is kept. * convert absolute alarm back to relative ([BMC #11233](http://bugs.meego.com/show_bug.cgi?id=11233)) Experiments show that at least Nokia phones (and thus perhaps also Mobical.com) interpret a fixed alarm as “repeat alarm with the same relative offset as on first occurrence”. The same transformation to relative alarm times is applied whenever the transformation to absolute alarm is enabled for a peer. * Sony Ericsson: enable conversion to absolute alarm times ([BMC #10092](http://bugs.meego.com/show_bug.cgi?id=10092)) Like Nokia and Mobical.net, Sony Ericsson phones also seem to be unable to deal with relative alarm times - verified with t700. * Sony Ericsson C510: workaround for SyncML violation The phone does not sent identifiers for the target database; using the source identifier as fallback allows a sync to run. * Fixed a regression affecting users who had created a config with SyncEvolution < 1.0. Using the config worked once, then failed with “No configuration for … found”. Users must manually remove the empty “peers” directory inside their affected configuration, the fix only makes configs without that directory usable again ([BMC #9381](http://bugs.meego.com/show_bug.cgi?id=9381)). * Removed obsolete workaround for older mKCal calendar storage. * Fixed error message in QtContacts backend. * Same SYNCEVOLUTION_DEBUG code as in master branch. * Some updates to synccompare, including a workaround for a Perl bug seen on Debian Testing with Perl 5.10.1-16 (Perl panic). * Fix compilation of syncevo-dbus-server with libnotify 0.7.0 ([BMC #10453](http://bugs.meego.com/show_bug.cgi?id=10453)). * Fixed compilation on Debian GNU/Hurd (no MAX_PATH, Mac OS X confusion). Source, Installation, Further information =============================== Source snapshots are in http://downloads.syncevolution.org/syncevolution/sources i386, amd64 and lpia binaries for Debian-based distributions are available via the “stable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:
debhttp://downloads.syncevolution.org/aptstablemain These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in http://downloads.syncevolution.org/syncevolution/evolution. In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
End of November, the scheduleworld.com service shut down. As a user [confirmed later](http://article.gmane.org/gmane.comp.mobile.syncevolution/1868), a notification had been sent earlier to paying customers. So the site is really gone and not just encountering some temporary problems. There does not seem to be any publicly accessible statement about the shutdown, hence this blog post. I have no information about the reasons for discontinuing the service. The notification to users did not give an explanation either. My guess is that the business side of the service did not work out. Syncing is a very challenging, technically difficult problem. I suspect that many users do not understand how much hard work goes into it and thus are less willing to pay for it. Then there is the competition with free services. If users are happy with online access Google Calendar, why should they bother paying for a synchronization service? There are reasons (see below), but perhaps not enough users care. This is really sad, for several reasons. ======================================== First, ScheduleWorld offered a technically superior SyncML implementation for calendar synchronization by supporting the full iCalendar 2.0 semantic, including [UID and RECURRENCE-ID](http://www.estamos.de/blog/2008/06/30/icalendar-20-detached-recurrences/). When describing these properties in 2008, I wrote that not supporting them “looks like a rather significant gap for SyncML-based calendar synchronization compared to proprietary solutions. Hopefully SyncML server vendors will wake up and do something about it…” I don’t think they have - or can anyone point to a SyncML server which supports this today, over one and a half year later? I’m not aware of any. Second, although ScheduleWorld used the open source Funambol server code, none of the extensions or modifications were ever made available. That’s perfectly compliant with the license of the code that was used (GPL instead of AGPL as today), but it implies that the ScheduleWorld source code is now lost and no-one can use or study it. Third, I believe that storing my data on my own hardware and synchronizing it via open standards is crucial. ScheduleWorld made that possible for users who did not want to run their own SyncML server. With true syncing instead of just online access, data is available while offline - pretty obvious, but worth spelling out explicitly. When syncing is based on open standards, users can decide which service they want to use. They don’t depend on that service to store their data; important, because any service might go away unexpectedly and take the data with it. With ScheduleWorld, all data was always and by design also stored locally, so no data was lost when the site shut down. The same is not true for many (if not all) of the now popular Web 2.0 services. Replacements for ScheduleWorld ============================== I’m not sure yet what I should recommend as replacement. Memotoo seems to be popular, but it does not support UID/RECURRENCE-ID semantic, so there are [issues when synchronizing complex iCalendar 2.0 calendar data](http://bugs.meego.com/show_bug.cgi?id=1180). For advanced users the [SyncEvolution server mode](http://syncevolution.org/development/http-server-howto) might be an alternative, but it is in a very rough state at this point and requires a publicly accessible machine for anytime, anywhere synchronization.
debhttp://downloads.syncevolution.org/aptstablemain These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in http://downloads.syncevolution.org/syncevolution/evolution. In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
The current development version of SyncEvolution, the one which will become 1.1, has some new features which may be useful for command line aficionados like myself: query and manipulate items in the databases that can be accessed with SyncEvolution backends. This is particularly useful for the new PIM storages in MeeGo, QtContacts and mkcal (formerly known as KCalExtended). Both are part of MeeGo Core, but come without any kind of frontend in Core. SyncEvolution provides a uniform way of testing these new storages using the command line or scripts. Installation ======== In MeeGo, make sure that at least 1.0.99.5 rpms (required for mkcal) or better, 1.0.99.6 (for QtContacts) get installed. The core “syncevolution” rpm is needed plus “syncevolution-kcalextended” resp. “syncevolution-qtcontacts”. When compiling from source, check out the master branch of [“syncevolution”](http://meego.gitorious.org/meego-middleware/syncevolution) and [“libsynthesis”](http://meego.gitorious.org/meego-middleware/libsynthesis) and follow the instructions in the “HACKING” document. Add –enable-kcalextended resp. –enable-qtcontacts as needed… HOWTO ======= These examples illustrate how to use the new options in combination with mkcal. In 1.0.99.6, “mkcal” can be used as type instead of “kcalextended”, as in the examples below. Only events are supported at the moment, todos and journal entries in the calendar are ignored. Wipe out calendar data the hard way:
rm-f~/.calendardb* Import an event:
$cat>/tmp/example.ics<<EOFBEGIN:VCALENDARPRODID:-//Ximian//NONSGMLEvolutionCalendar//ENVERSION:2.0METHOD:PUBLISHBEGIN:VEVENTDTSTART:20100823T170000DTEND:20100823T173000SUMMARY:firsteventCLASS:PUBLICEND:VEVENTEND:VCALENDAREOF$syncevolution--import/tmp/example.ics--source-propertytype=kcalextended@foobar#0:3f630f1d-8009-4465-907b-3ec28248b90b-rid[ERROR]stderr:kdedate/ksystemtimezone.cpp:313-cannotgetwall_clock_info(localzone)-QDBusError("org.freedesktop.DBus.Error.ServiceUnknown","Thenamecom.nokia.timewasnotprovidedbyany.servicefiles") Because many of these system libraries don’t expect to be used by command line users, they spew out a lot of debugging information. The SyncEvolution command line filters this output and suppresses most of it. Developers who *want* to see it, can set the SYNCEVOLUTION_DEBUG environment variable to 1. They should also add *–daemon=no* to run the operation inside the “syncevolution” process instead of the “syncevo-dbus-server”. The error in this case is [MeeGo bug](http://bugs.meego.com/show_bug.cgi?id=6174 “timed not started”), which I’ll strip from the output of the following examples. The command line above works without any “@foo” configuration and “bar” data source, because the source type is specified explicitly. Configuring this value permanently makes the command line shorter:
$syncevolution--configure--source-propertytype=kcalextended@defaultcalendar List all events:
$syncevolution--daemon=no--print-items@defaultcalendar3f630f1d-8009-4465-907b-3ec28248b90b-rid:firstevent Export events to stdout:
$syncevolution--export-@defaultcalendarBEGIN:VCALENDARPRODID:-//KDesktopEnvironment//NONSGMLlibkcal4.3//ENVERSION:2.0BEGIN:VEVENTCREATED:20100901T131100ZDTSTAMP:20100901T130859ZUID:a62893e3-3df1-4a92-b57b-d05c805d4078LAST-MODIFIED:20100901T131009ZSUMMARY:firsteventDTSTART:20100823T170000DTEND:20100823T173000TRANSP:OPAQUEEND:VEVENTEND:VCALENDAR Export one specific event into a file (like everything else, this also works for multiple events, writing into a single file or a directory):
$syncevolution--export/tmp/exported.ics@defaultcalendar3f630f1d-8009-4465-907b-3ec28248b90b-rid Update that event, showing of reading from stdin here:
`
$ perl -p -e ‘s/first event/first event modified/’ /tmp/example.ics | syncevolution –daemon=no –update - @default calendar 3f630f1d-8009-4465-907b-3ec28248b90b-rid
#0: 3f630f1d-8009-4465-907b-3ec28248b90b-rid
$ syncevolution –daemon=no –export - @default calendar | grep SUMMARY
SUMMARY:first event
Hmm,thatdidn'tworkasintended.Needtoinvestigate...ah,Ihaven'treimplementedthatpartwiththenewmkcalAPIyet.Willbefixedin1.0.99.6.Deletetheevent:
$ syncevolution –delete-items @default calendar 3f630f1d-8009-4465-907b-3ec28248b90b-rid
[INFO] calendar: deleting “first event”
REFERENCE===========
$ syncevolution –help
…
List items:
syncevolution –print-items
Export item(s):
syncevolution [–delimiter ] –export ||- [ …]
Add item(s):
syncevolution [–delimiter |none] –import ||-
Update item(s)
syncevolution –update
syncevolution [–delimiter |none] –update |- …
Remove item(s):
syncevolution –delete-items …
…
–print-items
Shows all existing items using one line per item using
the format “[: ]”. Whether the description
is available depends on the backend and the kind of data that it
stores.
SyncEvolution 1.0.1 ================ A bug fix release. The main reason for releasing it is that SyncEvolution 1.0 no longer worked on recent distros (Fedora Core 13, Debian testing) because of a name clash between the Bluez D-Bus utility code and recent glib. Details: * compile fix for FC 13 (and possibly others): use private copy of gdbus ([BMC #3556](http://bugs.meego.com/show_bug.cgi?id=3556)) * sync-ui: prevent overwriting device configs by accident ([BMC #3566](http://bugs.meego.com/show_bug.cgi?id=3566),[BMC #1194](http://bugs.meego.com/show_bug.cgi?id=1194)) Setting up a phone used the template name as config name and overwrote an existing configuration of another phone that was created using that same template. Now the code uses the Bluetooth device name as set on the device and checks for (less likely) collisions. It also sanitizes the name to avoid complicated config names (only relevant when also using the command line). * syncevo-dbus-server: accept ‘application/vnd.syncml+xml; charset=UTF-8’ for starting an HTTP session ([BMC #3554](http://bugs.meego.com/show_bug.cgi?id=3554)) The redundant charset specification was set by the Funambol Thunderbird client. Because of a literal comparison against ‘application/vnd.syncml+xml’ the messages were rejected. * config fix: operations on non-peer configs failed ([BMC #3157](http://bugs.meego.com/show_bug.cgi?id=3157)) When running operations on a non-peer configuration (like –restore @default addressbook), the operation fails with [ERROR] : type ‘select backend’ not supported * ZYB.com: service goes away end of June 2010, template removed ([BMC #3310](http://bugs.meego.com/show_bug.cgi?id=3310)) * some build ([BMC #2586](http://bugs.meego.com/show_bug.cgi?id=2586), [BMC #3557](http://bugs.meego.com/show_bug.cgi?id=3557)) and language updates Source, Installation, Further information =============================== Source snapshots are in http://downloads.syncevolution.org/syncevolution/sources i386, amd64 and lpia binaries of 1.0 for Debian-based distributions are available via the “stable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:
debhttp://downloads.syncevolution.org/aptstablemain These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in http://downloads.syncevolution.org/syncevolution/evolution. In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
After several betas and lot of testing, it’s finally time to announce the end of the 1.0 development cycle: SyncEvolution 1.0 is released and replaces 0.9.2 as the stable version. 0.1 was released over four years ago. It has always bee part of the long-term vision to bring “personal SyncML” to desktops. Thanks to the Synthesis engine and Intel’s support for the project, this goal has been reached and this release really deserves the magic 1.0 label. For those not familiar with the project, SyncEvolution synchronizes personal information management (PIM) data like contacts, calenders, tasks, and memos using the SyncML information synchronization standard. Up to and including 0.9.2, a third-party SyncML server was required. In 1.0, SyncEvolution itself is able to act as a SyncML server, both via HTTP and Bluetooth (direct sync with phones). SyncEvolution 1.0 ================= Major new features compared to previous stable release: * synchronize directly with a phone over Bluetooth/OBEX * accept Bluetooth/OBEX connections in cooperation with obexd >= 0.19 * run SyncEvolution as a rudimentary HTTP SyncML server The GTK sync-UI can be used to select a paired phone and create a configuration for it based on the bundled configuration templates. Configuration templates are included for Nokia phones; for other phones see the [HOWTO](http://syncevolution.org/development/sync-phone) and check out the Wiki there. Some users have already reported success for Sony Ericsson phones and added setup instructions. New templates from the Wiki can be dropped into ~/.config/syncevolution-templates under an arbitrary file name. Unexpected slow syncs can be detected when running as client ([MB #2416](http://bugzilla.moblin.org/show_bug.cgi?id=2416)) and unless turned off (see “preventSlowSync”), SyncEvolution aborts the session so that the situation can be analyzed. A refresh from client or server might be more suitable. The command line tool provides instructions at the end of its output. The GTK sync-UI points towards its recovery dialog. Automatic synchronization is supported by the syncevo-dbus-server ([MB #6378](http://bugzilla.moblin.org/show_bug.cgi?id=6378)). When that is installed, it will be started as part of a user session and keep running to trigger syncs in the background. Notifications are emitted when syncs start, end or fail ([MB #10000](http://bugzilla.moblin.org/show_bug.cgi?id=10000)). Automatic synchronization can be enabled separately for each peer (“autoSync=0/1”, off by default), will be done at regular intervals (“autoSyncInterval=30” minutes) when online long enough (“autoSyncDelay=5” minutes). That last option ensures that a) an automatic sync does not attempt to use a network connection unless it was already active and b) hopefully is also around long enough to complete the sync. The Synthesis XML configuration was split up into different parts which are assembled from /usr/share/syncevolution/xml. Files in ~/.config/syncevolution-xml override and extend the default files, which my be useful when adding support for a new phone. SyncML servers: * ZYB.com now works thanks to a workaround for anchor handling ([MB #2424](http://bugzilla.moblin.org/show_bug.cgi?id=2424)); only contacts tested because everything else is considered legacy by ZYB.com * Horde: avoid confusing the server with a deviceId that starts like the ones used in old Funambol clients, helps with calendar sync ([MB #9347](http://bugzilla.moblin.org/show_bug.cgi?id=9347)) * Mobical.net (and other, similar services): fix vCalendar 1.0 alarm properties before importing them ([MB #10458](http://bugzilla.moblin.org/show_bug.cgi?id=10458)) * desknow.com works when switching to SyncMLVersion = 1.1 * Funambol, Memotoo (and probably others): preserve meeting series when receiving update for detached recurrence ([MBC #1916](http://bugs.meego.com/show_bug.cgi?id=1916)) Evolution: * calendar backend: minor fix for change tracking when deleting a single instance of a recurring event * workaround for Evolution 2.30: “timezone cannot be retrieved because it doesn’t exist” is triggered incorrectly when importing non-standard timezone definitions because libecal changed an error code ([MB #9820](http://bugzilla.moblin.org/show_bug.cgi?id=9820)) Performance and reliability improvements ([MB #7708](http://bugzilla.moblin.org/show_bug.cgi?id=7708)): * synccompare much faster * database dumps consume less disk space * more intelligent about expiring obsolete session directories and backups * database accesses are reduced in several backends * shorter logs ([MB #8092](http://bugzilla.moblin.org/show_bug.cgi?id=8092)) * message resending helps under unreliable network connectivity (“RetryInterval”) * full support for suspend&resume in SyncEvolution client to SyncEvolution or Synthesis server syncs * better handling of certain third-party time zone definitions ([MBC #1332](http://bugs.meego.com/show_bug.cgi?id=1332)) Improved GTK sync-UI: * revised config screen: all in one list where entries can be expanded, integrated setup of sync with other devices * recovery support: restore from backup, unexpected slow sync handling * spinner while network is in use ([MB #2229](http://bugzilla.moblin.org/show_bug.cgi?id=2229)) * interactive password requests ([MB #6376](http://bugzilla.moblin.org/show_bug.cgi?id=6376)) * uses new D-Bus API Command line: * fixed printing of rejected items ([MB #7755](http://bugzilla.moblin.org/show_bug.cgi?id=7755)) * consistent logging of added/updated/deleted items with short description * improved error reporting (textual descriptions instead of plain error codes [MB #2069](http://bugzilla.moblin.org/show_bug.cgi?id=2069), partial success [MB #7755](http://bugzilla.moblin.org/show_bug.cgi?id=7755), record and show first ERROR encountered [MB #7708](http://bugzilla.moblin.org/show_bug.cgi?id=7708)) * can create new sources ([MB #8424](http://bugzilla.moblin.org/show_bug.cgi?id=8424)) * runs operations inside daemon and thus avoids conflicts with operations done by other clients; for testing purposes (like running a client which talks to a local server in the daemon) it is still possible to ignore the daemon (–daemon=no, [MB #5043](http://bugzilla.moblin.org/show_bug.cgi?id=5043)) * revised README, now also available as man page ([MBC #690](http://bugs.meego.com/show_bug.cgi?id=690)) [Redesigned](http://syncevolution.org/development/direct-synchronization-aka-syncml-server) and reimplemented D-Bus API, used by sync-UI and command line: * central syncevo-dbus-server controls configurations and sync sessions * accepts incoming SyncML connection requests and messages received by independent transport stubs (obexd, HTTP server, …) * can be used by multiple user interfaces at once * fully documented, see src/dbus/interfaces and http://api.syncevolution.org * no longer depends on dbus-glib with hand-written glue code for C++, instead uses gdbus plus automatic C++ binding generated via C++ templates Revised [configuration layout](http://syncevolution.org/development/configuration-handling “design document for configuration handling”) ([MB #8048](http://bugzilla.moblin.org/show_bug.cgi?id=8048)): * several peer-independent sync and source properties are shared between multiple peers * they can be accessed without selecting a specific peer, by using an empty config name or with the new “@” syntax * user interface of command line unchanged * old configurations can be read and written, without causing unwanted slow syncs when moving between stable and unstable SyncEvolution versions * old configurations can be migrated with the “–migrate” command line switch; however, then older SyncEvolution can no longer access them and migrating more than one old configuration causes the second or later configuration to loose its “deviceId” property (which is shared now), causing a slow sync once * config names may contain characters that are not allowed in the file names used for the underlying files; will be replaced with underscores automatically ([MB #8350](http://bugzilla.moblin.org/show_bug.cgi?id=8350)) Upgrading from 0.9.x: * Upgrading and downgrading should work seamlessly when using existing configurations. * The new configuration layout is only used when creating new configurations or explicitly invoking “syncevolution –migrate” (see above). Such configs cannot be used by older SyncEvolution releases. * The new “RetryInterval” property causes messages to be resent after 2 minutes (increased from 1 minute in previous 1.0 betas). At least the Funambol server is known to not handle this correctly in all [cases](http://funzilla.funambol.com/show_bug.cgi?id=7910). So in the Funambol config template the interval is set to zero, disabling the feature. Disabling the feature must be done manually in existing Funambol configurations. SyncEvolution 1.0 beta 3 -> 1.0 final ===================================== Bug fixes and new features: * Configuration templates are stored in a single file ([MBC #1208](http://bugs.meego.com/show_bug.cgi?id=1208)). New templates (like something downloaded from [the Wiki](http://syncevolution.org/wiki) can be dropped into $HOME/.config/syncevolution-templates using an arbitrary file name. * Progress and per-source status are now also reported and recorded when running in server mode ([MBC #1359](http://bugs.meego.com/show_bug.cgi?id=1359)). There are still several limitations (sync mode not reported, no information about sent/received/processed items while the sync runs, see [MBC #2786](http://bugs.meego.com/show_bug.cgi?id=2786)). * Better handling of certain third-party time zone definitions ([MBC #1332](http://bugs.meego.com/show_bug.cgi?id=1332)). Better logging to track down such problems. * D-Bus server + command line: return error code when failed ([MBC #2193](http://bugs.meego.com/show_bug.cgi?id=2193)) * syncevo-phone-config: simplified command line options, several bug fixes (syntax error, incorrect handling of calendar+todo, [MBC #1197](http://bugs.meego.com/show_bug.cgi?id=1197)) * Revised README, now also available as man page ([MBC #690](http://bugs.meego.com/show_bug.cgi?id=690)). Conversion of D-Bus API documentation into .html page ([MBC #1745](http://bugs.meego.com/show_bug.cgi?id=1745)). * Funambol, Memotoo (and probably others): preserve meeting series when receiving update for detached recurrence ([MBC #1916](http://bugs.meego.com/show_bug.cgi?id=1916)) * Fix for potential out-of-bounds memory access ([MBC #1007](http://bugs.meego.com/show_bug.cgi?id=1007)). * HTTP server: fix for potential crash when second session was requested while an older one was still running, initial sync was done without libical time zone information and thus may have mismatched times ([MBC #2435](http://bugs.meego.com/show_bug.cgi?id=2435)) * Nokia E55: convert alarm times ([MBC #1657](http://bugs.meego.com/show_bug.cgi?id=1657)). This is done via a new remote rule in /usr/share/syncevolution/xml/remoterules/server/46_E55.xml If another phone needs the same treatment, then copy that file to ~/.config/syncevolution-xml/remoterules/server and edit the element. * GTK GUI: styling fix ([MBC #1372](http://bugs.meego.com/show_bug.cgi?id=1372)), updated toolbar for MeeGo 1.0 ([MBC #1970](http://bugs.meego.com/show_bug.cgi?id=1970)), avoid duplicating configs when selecting a config created by syncevo-phone-config or the command line ([MBC #1266](http://bugs.meego.com/show_bug.cgi?id=1266)), scroll bars for emergency window ([MBC #1296](http://bugs.meego.com/show_bug.cgi?id=1296)), avoid compile problem on Fedora Core 13 due to name collision with system sync() call, updated translations. Known Issues ============ Compatibility with phones has not been tested as well as compatibility with the officially supported SyncML servers. Some issues have been reported which still need to be investigated: * Nokia N85: ignores refresh-from-server? ([MBC #2722](http://bugs.meego.com/show_bug.cgi?id=2722)) * Sony Ericsson W595: All-day-events created in evolution are synchronized as all-day-events + 1 additional day in the mobile ([MBC #2093](http://bugs.meego.com/show_bug.cgi?id=2093)) * Nokia N81: Evolution contacts loose “Other” email-addresses when synced two-way ([MBC #2566](http://bugs.meego.com/show_bug.cgi?id=2566)) * Nokia phones: absolute alarm time? ([MBC #1657](http://bugs.meego.com/show_bug.cgi?id=1657)) \**Call for action**: [test with your phone](http://syncevolution.org/development/sync-phone “Phone Sync HOWTO”) and [report](http://syncevolution.org/wiki/phone-compatibility-template “Phone Compatibility Wiki page”) which config works and how well synchronization works. Other known issues: * server progress events: no information about sync mode ([MBC #2786](http://bugs.meego.com/show_bug.cgi?id=2786)) * Calendar event alarm synchronization between N900 and Goosync ([MBC #2764](http://bugs.meego.com/show_bug.cgi?id=2764)) Source, Installation, Further information =============================== Source snapshots are in http://downloads.syncevolution.org/syncevolution/sources i386, amd64 and lpia binaries of 1.0 for Debian-based distributions are available via the “stable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:
debhttp://downloads.syncevolution.org/aptstablemain These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in http://downloads.syncevolution.org/syncevolution/evolution. In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
SyncEvolution 1.0 beta 3 is available. This release is feature complete and has been tested thoroughly, so only minor bug fixes are expected before releasing 1.0. Beta 3 is ready for day-to-day use and for getting packaged in staging distros as replacement for 0.9.2 or previous betas. If you find issues, please [report them](http://syncevolution.org/support). Binary packages are provided, including Bluetooth support. Major improvements:
automatic, time-driven synchronization in the background
SyncEvolution 0.9.2 and 1.0 beta are available. 0.9.2 replaces 0.9.1 as the official stable release. Binary packages are provided. Changes in both releases are listed below. 1.0 beta contains all changes included in 0.9.2. Development has already shifted towards 1.0, but if you find issues in either version, please report them. In the 1.0 beta, SyncEvolution itself is already able to act as a SyncML server, both via HTTP and Bluetooth (direct sync with phones). Because of incompatible versions of libbluetooth in different distros, the precompiled 1.0 beta packages do not include Bluetooth support. One has to compile from source to use that. **2010-01-30:** if you had problems installing the 1.0 beta binaries because of dependencies on libbluetooth2 or libpcre.so.3, then please try again. Binaries without these dependencies replaced the initial set of binaries today (-3 release instead of -2 for .rpm and .deb). To learn more about SyncEvolution, come to **FOSDEM 2010**:
SyncEvolution 0.9.1 is available. It replaces 0.9 as the official stable release. Changes since that version are listed below. Changes made between beta releases (for those who helped testing them) are documented in the beta release announcement. There are some known issues, see below. The more issues get reported and fixed, the sooner there will be a 0.9.2 maintenance update, so keep the reports coming. If you want to report something or get involved, then contact the team. SyncEvolution synchronizes personal information management (PIM) data like contacts, calenders, tasks, and memos using the SyncML information synchronization standard. Up to and including 0.9.x, a third-party SyncML server is required. In the current “master” branch, SyncEvolution itself is already able to act as a SyncML server. Support for direct synchronization via Bluetooth is planned for 1.0. A snapshot release will be made available as soon as possible - end of October obviously did not work out…
Non-consecutive header level increase; H1 to H3 [myst.header]
SyncEvolution 0.9.1 beta 2 is available, release after fixing all issues which were found in beta 1. It is going to replace 0.9 as the official stable release soon, so this is the chance to find and report problems before they sneak into the final 0.9.1 - die, bugs, die!
Non-consecutive header level increase; H1 to H3 [myst.header]
SyncEvolution 0.9 has been released! The 0.9 version replaces 0.8.1 as the stable version on Linux desktops. Mac OS X and Maemo have not been updated and remain at 0.8.1 (hint: volunteers wanted). 0.9 binaries in .deb, .tar.gz and (for the first time) .rpm format are provided for x86 in 32 and 64 bit mode. Moblin 2.0 comes with SyncEvolution included in the normal Moblin image, with updates provided via the package repositories. Planning and work for SyncEvolution 1.0 is in full swing. 1.0 is intended to add the SyncML server role for direct synchronization with other devices. If you want to get involved, then contact the team: http://syncevolution.org/support
Non-consecutive header level increase; H1 to H3 [myst.header]
This post documents known issues in SyncEvolution 0.9 beta 3 and workarounds. [Update 24.07.2009] The 0.9 beta3 20090723 hotfix solves these issues, so make sure your SyncEvolution is up-to-date.
#4676: error messages caused by incorrect usage of the command line (like typos in the property names) are not printed anymore. Correct command lines work as intended, but both new users (experimenting) and old users (typos) are likely to make mistakes occasionally. No workaround, needs to be fixed as soon as possible. Run with SYNCEVOLUTION_DEBUG=1 in your environment to see the error messages again. #4551: SSL certificate check with libsoup/gnutls currently fails for Google. The last-minute workaround for the SyncEvolution Google configuration template was to use http instead of https. It turned out that Google does not provide SyncML without SSL. That means that the default configuration has to be modified. This cannot be done via the GUI. Set up the configuration either via GUI or command line. Then use the command line to update the configuration:
syncevolution--configure\--sync-propertysyncURL=https://m.google.com/syncml\--sync-propertySSLVerifyHost=0\--sync-propertySSLVerifyServer=0\google
The end is near - SyncEvolution 0.9 is almost done. For the first time in the 0.9 series, precompiled binaries are made available again together with the new 0.9 beta 3 source snapshot. Users are encouraged to upgrade now and give feedback before the final 0.9 release. See below for upgrade and install instructions. The initial 0.9 beta 3 packages had
two problems which were discovered while testing them. Make sure that you install the updated 0.9 beta 3 20090723 version.
SyncEvolution 0.9 beta 3 is released and I still haven’t found the time to try out whether the 0.9 series still compiles on Maemo and Mac OS X. Unless someone with some skills in compiling software on those platforms and the necessary motivation steps up, these platforms might not be supported in the initial 0.9 release. The HTTP transport from 0.8, libcurl, is still supported and compiled regularly. Internal API changes were made in the Mac OS X backend as part of the normal maintenance, without testing them yet. So chances are that it will compile and run without too much effort. If you are interested, the HACKING document contains sections on how previous releases were compiled - give it a try and help out fellow users by sharing the result!
Update 02.11.2009: a mail asking for help was sent to the Maemo developers list. The Mac OS X port requires a bit more changes, but there has been some progress. A permanent maintainer would still be needed, though.
The SyncEvolution team is pleased to announce our new web site and place to interact with the SyncEvolution community. Most of the content and documentation from our previous site (http://estamos.de) has been migrated here. Let us know what you think!
Non-consecutive header level increase; H1 to H3 [myst.header]
This release modernizes the code base and removes usage of out-dated
libraries and APIs. All Python scripts require Python 3. The major
version bump reflects that this release is not just a minor bug fix.
However, no new features were added.
Binaries on syncevolution.org get built for distros >= Ubuntu Bionic
and Debian Buster. Testing is now based on Docker containers instead
of a custom schroot solution, so adding testing against other distros
will be easier in the future. Compilation on Fedora Rawhide was
already added.
Maintenance release. syncevolution.org binaries are now getting
compiled for distros >= Ubuntu Xenial 16.04 LTS. Usage of deprecated
libraries (GNOME keyring) and APIs (SoupAsyncSession) was
replaced. libical v3 is supported.
The code now compiles more cleanly with recent compilers and depends
on C++11 support.
Maintenance release. syncevolution.org binaries are now getting compiled for distros >= Ubuntu Trusty 14.04 LTS, which allowed removing several hacks that were needed when building binaries that also had to run on older distros. Compilation from source for old distros should still work as before, but is not getting tested anymore. Compile problems with recent libraries (libical v2) and tools (GCC v6) were resolved. Syncing via Bluetooth with certain phones now should work reliably in incremental mode. New backends for the Trinity Desktop Environment (TDE) were added to the source code. Details: * ObexTransportAgent.cpp: properly shut down connection (FDO #91485) Apparently there’s a race condition in the OBEX transport that causes the connection to phones via Bluetooth to be shut down prematurely. Some phones react by doing a slow sync instead of an incremental sync the next time. * support non-readable parent directories (FDO #91000) The previous mkdir_p() walked down top to bottom and checked each path entry as it went along. That approach failed unnecessarily when some existing parent directory could not be read (non-readable /home, for example). * avoid using dbus-launch (Debian #836399) dbus-launch is considered deprecated because of the X11 dependency. See https://lists.debian.org/debian-devel/2016/08/msg00554.html “Mass bug filing: use and misuse of dbus-launch (dbus-x11)” The dbus-session.sh script still needs to start the D-Bus daemon when used in the nightly testing, so the code now does it by invoking the dbus-daemon directly. syncevo-http-server still has some usage of dbus-launch left, but that’s strictly for systems which don’t have the more modern D-Bus. * syncevo-dbus-server integrates better with systemd (FDO #92164) A .service file allows the D-Bus daemon to start the service via systemd, thus ensuring that the process environment is correct. Patch from Simon McVittie. Auto-starting as part of the desktop login uses D-Bus activation if the “dbus-send” tool is installed. * syncevolution.org: compile on Ubuntu Trusty, libical v1/v2 compatibility syncevolution.org binaries are now getting compiled on Ubuntu Trusty and thus no longer support distros with older EDS. The code should still compile against older EDS (for example, for Maemo), but that is not getting tested anymore. This allows removing the dynamic linker hacks related to older libraries, which was only used in those binaries. Instead, backends using libical or EDS get compiled on Ubuntu Trusty and then the soname of those libs get patched to make the backend module usable in combination with a different set of libs. That patching is part of a script maintained in the syncevolution.org build infrastructure. This approach was already used before to generate different EDS backends for EDS versions with the newer EClient API, because that turned out to be easier than the dynamic loading approach. It works because none of the methods used by SyncEvolution changed their ABI, only some other parts of the libraries did. Should there ever be a situation again that cannot be handled like this, then backends also get compiled on different distros than Ubuntu Trusty (for example, the ActiveSync backend for Debian Stretch is built on Debian Stretch). libical still requires one special hack: system time zone loading in libical v1 (and only in that version, v2 has builtin support again) must be overridden such that time zones are generated with rules instead of transitions because that is more compatible with the peers that SyncEvolution exchanges data with. That hack now relies on overriding the two relevant functions inside the main binaries (has to be there, otherwise libical still ends up calling its own internal implementation). The overriding code is in libsyncevo-icaltz-util.so.0 and depends on libical.so.1. If libsyncevo-icaltz-util.so.0 can be loaded, the wrappers in the main binary use it, otherwise they fall through to the code from the current libical.so, which then should be libical.so.2 or more recent. This hack is active by default when libical v1 is detected during configuration. * optionally show debug output in –version output SYNCEVOLUTION_DEBUG=1 syncevolution –daemon=no –version now dumps also the debug information gathered by the binary compatibility code. It was only available in sync logs before. * various build fixes for libical v2, GCC v6/C++14 Upgrading from releases <= 1.3.99.4: ———————————— If the value of “username/databaseUser/proxyUser” contains a colon, the “user:” prefix must be added to the value, to continue treating it like a plain user name and not some reference to an unknown identity provider (like “id:”, “goa:”, “signon:”, etc.). The lookup of passwords in GNOME Keyring was updated slightly in 1.3.99.5. It may be necessary to set passwords anew if the old one is no longer found. Upgrading from release 1.2.x: —————————– The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts): syncevolution –configure \ syncFormat=text/x-vcard \ mobical addressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually: syncevolution –configure \ enableRefreshSync=TRUE \ funambol Upgrading from releases before 1.2: ———————————– Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Source, Installation, Further information ========================================= Source code bundles for users are available in https://download.01.org/syncevolution/syncevolution/sources and the original source is in the git repositories http://cgit.freedesktop.org/SyncEvolution/ i386 and amd64 binaries for Debian-based distributions are available via the “stable” syncevolution.org repository. Add the following entry to your /etc/apt/source.list:
The GPG key for the repository needs to be imported as root with:
Maintenance release. Binaries now also get compiled for Debian 8.0 “Jessie”. The apt repositories on syncevolution.org (mirrored on download.01.org) are signed with a different key than before, see “Installation” below. Details: * avoid time zone issue with Funambol server The Funambol iCalendar 2.0 parser fails to handle time zones with quotation marks around the TZID value, which is something that SyncEvolution started to add in 1.4.99.3. While it is valid to quote like that, it is not necessary, so avoid quoting in this case to restore interoperability. * syncevo-http-server: stop using deprecated twisted.web.error (FDO #90419) This has become a real problem for example on Fedora 22 where the old name is no longer available. * syncevo-http-server: use TLS instead of SSLv3 This fixes a potential security risk and connection problems with clients that don’t support SSLv3 anymore. * syncing: avoid segfault for invalid text inside items (FDO #90118) As reported by Canonical, syncing fails if data items contain text which is not correct UTF-8 in one of the fields that SyncEvolution logs in the command line output (like SUMMARY of a calendar event). That is because the byte string coming from the item is passed unchecked to the D-Bus implementation for transmission via D-Bus. But D-Bus strings must be correct UTF-8, so depending on the D-Bus library in use, one gets a segfault (GIO D-Bus, due to an unchecked NULL pointer access) or an “out of memory” error (libdbus, which checks for NULL). SyncEvolution now replaces invalid bytes with a question mark in its output while preserving the rest of the text. * file backend: log item manipulation Extracting a meaningful description of each item from the Synthesis engine when updating and adding items is easy to do for items of certain known types (contacts and calendar items). * command line: preserve log prefix of target side of local sync In some cases, the prefix which was supposed to be embedded in the log messages from the target side of a local sync got lost on the way to the command line tool. Primarily this affected the added/updated/deleted messages, as in: [INFO remote@client] @client/addressbook: started [INFO remote@client] updating “Joan Doe” [INFO remote@client] @client/addressbook: received 1/1 * compile fix: use ${PKG_CONFIG} instead of pkg-config. This fixes the build on Exherbo that only has prefixed versions of pkg-config. * WebDAV: handle 403 during Google OAuth authentication When sending an access token with insufficient scope (for example, because the Ubuntu Online Accounts service definition was incomplete, as documented in FDO #86824), Google responds with a 403 “service denied” error. This is now dealt with by retrying, just as for a transient 401 error. * CalDAV: more efficient “is empty” check (FDO #86335) Since 1.4.99.4, syncing WebDAV collections always checks first whether there are items in the collections. This was partly done for slow sync prevention (which is not necessary for empty collections), partly for the “is the datastore usable” check. However, this did not take into account that for CalDAV collections, the entire content gets downloaded for this check. That is because filtering by item type (VEVENT vs. VJOURNAL) is not implemented correctly by all servers. So now all CalDAV syncs, whether incremental or slow, always transfered all items, which is not the intention (incremental syncs should be fast and efficient). This release adds a more efficient isEmpty() check: for simple CardDAV collections, only luid and etag get transferred, as in listAllItems(). This is the behavior from 1.5. For CalDAV, a report with a filter for the content type is used and the transfer gets aborted after the first item, without actually double-checking the content of the item. This is different from listAllItems(), which really transfers the content. This extra content check would only be needed for some old servers (Radical 0.7) and is not essential, because reporting “not empty” even when empty is safe. * WebDAV: send Basic Auth via http in some cases (FDO #57248) It turned out that finding databases on an Apple Calendar server accessed via http depends on sending Basic Auth even when the server does not ask for it: without authentication, there is no information about the current principal, which is necessary for finding the user’s databases. To make this work again, sending the authentication header is now forced for plain http if (and only if) the request which should have returned the principal URL fails to include it. This implies sending the same request twice, but as this scenario should be rare in practise (was only done for testing), this is acceptable. * Ubuntu Online Accounts: support plain text credentials The backend for UOA was rewritten by Alberto Mardegan and now also can use plain username/password credentials stored in UOA. * various compiler error and warning fixes Upgrading from releases <= 1.3.99.4: ———————————— If the value of “username/databaseUser/proxyUser” contains a colon, the “user:” prefix must be added to the value, to continue treating it like a plain user name and not some reference to an unknown identity provider (like “id:”, “goa:”, “signon:”, etc.). The lookup of passwords in GNOME Keyring was updated slightly in 1.3.99.5. It may be necessary to set passwords anew if the old one is no longer found. Upgrading from release 1.2.x: —————————– The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts): syncevolution –configure \ syncFormat=text/x-vcard \ mobical addressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually: syncevolution –configure \ enableRefreshSync=TRUE \ funambol Upgrading from releases before 1.2: ———————————– Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Source, Installation, Further information ========================================= Source code bundles for users are available in https://download.01.org/syncevolution/syncevolution/sources and the original source is in [the git repositories](http://cgit.freedesktop.org/SyncEvolution/). i386, lpia and amd64 binaries for Debian-based distributions are available via the “stable” syncevolution.org repository. Add the following entry to your /apt/source.list:
debhttps://download.01.org/syncevolution/aptstablemain The GPG key for the repository needs to be imported as root with:
apt-keyadv--keyserverkeyserver.ubuntu.com--recv-keysB2EC3981 Then install “syncevolution-evolution”, “syncevolution-kde” and/or “syncevolution-activesync”. These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 10.04 LTS (Lucid), except for ActiveSync binaries which were compiled for Debian Wheezy, Ubuntu Saucy and Ubuntu Trusty, and Debian Jessie. The packages mentioned above are meta-packages which pull in suitable packages matching the distro during installation. Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](https://download.01.org/syncevolution/syncevolution/). In contrast to 0.8.x archives, the 1.x .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps. More specific [HOWTOs](/wiki/howto) can be found in the Wiki.
About 1.5 ========= This version is the new stable, supported version of SyncEvolution. Compared to the last 1.4.99.4 pre-release only minor changes were done (see below). This section summarizes the changes since the last stable release, 1.4.1. Based on community feedback and discussions, the terminology used in SyncEvolution for configuration, local sync and database access was revised. Some usability issues with setting up access to databases were addressed. Interoperability with WebDAV servers and in particular Google Contacts was enhanced considerably. Access to iCloud contacts was reported as working when using username=foobar@icloud.com and password, but is not formally tested. Syncing with iCloud calendars ran into a server limitation (reported as 17001498 “CalDAV REPORT drops calendar data”) and needs further work ([FDO #72133](https://bugs.freedesktop.org/show_bug.cgi?id=72133)). Contact data gets converted to and from the format typically used by CardDAV servers, so now anniversary, spouse, manager, assistant and instant message information are exchanged properly. Custom labels get stored in EDS as extensions and no longer get lost when updating some other aspects of a contact. However, Evolution does not show custom labels and removes them when editing a property which has a custom label ([BGO #730636](https://bugzilla.gnome.org/show_bug.cgi?id=730636)). Scanning for CardDAV/CalDAV resources was enhanced. It now finds additional calendars with Google CalDAV. For Google, the obsolete SyncML config template was removed and CalDAV/CardDAV were merged into a single “Google” template. Using Google Calendar/Contacts with OAuth2 authentication on a headless server becomes a bit easier: it is possible to set up access on one system with a GUI using either gSSO or GNOME Online Accounts, then take the OAuth2 refresh token and use it in SyncEvolution on a different system. See [the oauth2 backend README](http://cgit.freedesktop.org/SyncEvolution/syncevolution/tree/src/backends/oauth2/README) for details. syncevolution.org binaries do not include this feature. The PIM Manager API also supports Google Contact syncing. Some problems with suspending a PBAP sync were fixed. Suspend/abort can be tested with the sync.py example. Performance is better for local syncs and PBAP caching. The most common case, a two-way sync with no changes on either side, no longer rewrites any meta data files. CPU consumption during local sync was reduced to one third by exchanging messages via shared memory instead of internal D-Bus. Redundant vCard decode/encode on the sending side of PBAP and too agressive flushing of meta data during a normal sync were removed. The EDS memo backend is able to switch between syncing in plain text and iCalendar 2.0 VJOURNAL automatically. Graham Cobb fixed some all-day conversion issues in activesyncd. The updated version is part of the 1.5 release on syncevolution.org. Details: * source -> datastore rename, improved terminology The word “source” implies reading, while in fact access is read/write. “datastore” avoids that misconception. Writing it in one word emphasizes that it is single entity. While renaming, also remove references to explicit –*-property parameters. The only necessary use today is “–sync-property ?” and “–datastore-property ?”. –datastore-property was used instead of the short –store-property because “store” might be mistaken for the verb. It doesn’t matter that it is longer because it doesn’t get typed often. –source-property must remain valid for backward compatility. As many user-visible instances of “source” as possible got replaced in text strings by the newer term “datastore”. Debug messages were left unchanged unless some regex happened to match it. The source code will continue to use the old variable and class names based on “source”. Various documentation enhancements: Better explain what local sync is and how it involves two sync configs. “originating config” gets introduces instead of just “sync config”. Better explain the relationship between contexts, sync configs, and source configs (“a sync config can use the datastore configs in the same context”). An entire section on config properties in the terminology section. “item” added (Todd Wilson correctly pointed out that it was missing). Less focus on conflict resolution, as suggested by Graham Cobb. Fix examples that became invalid when fixing the password storage/lookup mechanism for GNOME keyring in 1.4. The “command line conventions”, “Synchronization beyond SyncML” and “CalDAV and CardDAV” sections were updated. It’s possible that the other sections also contain slightly incorrect usage of the terminology or are simply out-dated. * local sync: allow config name in syncURL=local:// Previously, only syncURL=local://@ was allowed and used the “target-config@context name” config as target side in the local sync. Now “local://config-name@context-name” or simply “local://config-name” are also allowed. “target-config” is still the fallback if only a context is give. It also has one more special meaning: “–configure target-config@google” will pick the “Google” template automatically because it knows that the intention is to configure the target side of a local sync. It does not know that when using some other name for the config, in which case the template (if needed) must be specified explicitly. The process name in output from the target side now also includes the configuration name if it is not the default “target-config”. * command line: revise usability checking of datastores When configuring a new sync config, the command line checks whether a datastore is usable before enabling it. If no datastores were listed explicitly, only the usable ones get enabled. If unusable datastores were explicitly listed, the entire configure operation fails. This check was based on listing databases, which turned out to be too unspecific for the WebDAV backend: when “database” was set to some URL which is good enough to list databases, but not a database URL itself, the sources where configured with that bad URL. Now a new SyncSource::isUsable() operation is used, which by default just falls back to calling the existing Operations::m_isEmpty. In practice, all sources either check their config in open() or the m_isEmpty operation, so the source is usable if no error is enountered. For WebDAV, the usability check is skipped because it would require contacting a remote server, which is both confusing (why does a local configure operation need the server?) and could fail even for valid configs (server temporarily down). The check was incomplete anyway because listing databases gave a fixed help text response when no credentials were given. For usability checking that should have resulted in “not usable” and didn’t. The output during the check was confusing: it always said “listing databases” without giving a reason why that was done. The intention was to give some feedback while a potentially expensive operation ran. Now the isUsable() method itself prints “checking usability” if (and only if!) such a check is really done. Sometimes datastores were checked even when they were about to be configure as “disabled” already. Now checking such datastores is skipped. * command line: fix –update from directory The “–update
This is the first release candidate for 1.5. No further changes are planned except for fixing yet-to-be-discovered bugs - so find them now! :-) One focus in this release was on minimizing CPU consumption and disk writes. The most common case, a two-way sync with no changes on either side, no longer rewrites any meta data files. CPU consumption during local sync was reduced to one third by exchanging messages via shared memory instead of internal D-Bus. Redundant vCard decode/encode on the sending side of PBAP and too agressive flushing of meta data during a normal sync were removed. Altogether, sending 1000 contacts with photo data in a refresh-from-server local sync takes only one sixth of the CPU cycles compared to 1.3.99.3 (measured with valgrind’s callgrind on x86_64). Based on community feedback and discussions, the terminology used in SyncEvolution for configuration, local sync and database access was revised. Some usability issues with setting up access to databases were addressed. For Google, the obsolete SyncML config template was removed and CalDAV/CardDAV were merged into a single “Google” template. Using Google Calendar/Contacts with OAuth2 authentication on a headless server becomes a bit easier: it is possible to set up access on one system with a GUI using either gSSO or GNOME Online Accounts, then take the OAuth2 refresh token and use it in SyncEvolution on a different system. See the [new OAuth2 backend README](http://cgit.freedesktop.org/SyncEvolution/syncevolution/tree/src/backends/oauth2/README). Some issues accessing Apple iCloud were fixed such that CardDAV works by just giving SyncEvolution username=foobar@icloud.com and password. No throrough testing was done, so iCloud support is still experimental. The PIM Manager API also supports Google Contact syncing. Some problems with suspending a PBAP sync were fixed. Suspend/abort can be tested with the sync.py example. The EDS memo backend is able to switch between syncing in plain text and iCalendar 2.0 VJOURNAL automatically. Details: ——— * oauth2: new backend using libsoup/libcurl New backend implements identity provider for obtaining OAuth2 access token for systems without HMI support. Access token is obtained by making direct HTTP request to OAuth2 server and using refresh token obtained by user in some other way. New provider automatically updates stored refresh token when OAuth2 server is issuing new one. * PBAP: use raw text items This avoids the redundant parse/generate step on the sending side of the PBAP sync. * datatypes: raw text items with minimal conversion ([FDO #52791](https://bugs.freedesktop.org/show_bug.cgi?id=52791)) When using “raw/text/calendar” or “raw/text/vcard” as SyncEvolution “databaseFormat”, all parsing and conversion is skipped. The backend’s data is identical to the item data in the engine. Finding duplicates in a slow sync is very limited when using these types because the entire item data must match exactly. This is useful for the file backend when the goal is to store an exact copy of what a peer has or for limited, read-only backends (PBAP). The downside of using the raw types is that the peer is not given accurate information about which vCard or iCalendar properties are supported, which may cause some peers to not send all data. * engine: flush map items less frequently The Synthesis API does not say explicitly, but in practice all map items get updated in a tight loop. Rewriting the m_mappingNode (case insensitive string comparisons) and serialization to disk (std::ostrstream) consume a significant amount of CPU cycles and cause extra disk writes that can be avoided by making some assumptions about the sequence of API calls and flushing only once. * SoupTransport: drop CA file check It used to be necessary to specify a CA file for libsoup to enable SSL certificate checking. Nowadays libsoup uses the default CA store unless told otherwise, so the check in SyncEvolution became obsolete. However, now there is a certain risk that no SSL checking is done although the user asked for it (when libsoup is not recent enough or compiled correctly). * local sync: exchange SyncML messages via shared memory Encoding/decoding of the uint8_t array in D-Bus took a surprisingly large amount of CPU cycles relative to the rest of the SyncML message processing. Now the actual data resides in memory-mapped temporary files and the D-Bus messages only contain offset and size inside these files. Both sides use memory mapping to read and write directly. For caching 1000 contacts with photos on a fast laptop, total sync time roughly drops from 6s to 3s. To eliminate memory copies, memory handling in libsynthesis or rather, libsmltk is tweaked such that it allocates the buffer used for SyncML message data in the shared memory buffer directly. This relies on knowledge of libsmltk internals, but those shouldn’t change and if they do, SyncEvolution will notice (“unexpected send buffer”). * local sync: avoid updating meta data when nothing changed The sync meta data (sync anchors, client change log) get updated after a sync even if nothing changed and the existing meta data could be used again. This can be skipped for local sync, because then SyncEvolution can ensure that both sides skip updating the meta data. With a remote SyncML server that is not possible and thus SyncEvolution has to update its data. This optimization is only used for local syncs with one source. It is based on the observation that when the server side calls SaveAdminData, the client has sent its last message and the sync is complete. At that point, SyncEvolution can check whether anything has changed and if not, skip saving the server’s admin data and stop the sync without sending the real reply to the client. Instead the client gets an empty message with “quitsync” as content type. Then it takes shortcuts to close down without finalizing the sync engine, because that would trigger writing of meta data changes. The server continues its shutdown normally. This optimization is limited to syncs with a single source, because the assumption about when aborting is possible is harder to verify when multiple sources are involved. * PIM: include CardDAV in CreatePeer() This adds “protocol: CardDAV” as a valid value, with corresponding changes to the interpretation of some existing properties and some new ones. The API itself is not changed. Suspending a CardDAV sync is possible. This freezes the internal SyncML message exchange, so data exchange with the CardDAV server may continue for a while after SuspendPeer(). Photo data is always downloaded immediately. The “pbap-sync” flag in SyncPeerWithFlags() has no effect. Syncing can be configured to be one-way (local side is read-only cache) or two-way (local side is read/write). Meta data must be written either way, to speed up caching or allow two-way syncing. The most common case (no changes on either side) will have to be optimized such that existing meta data is not touched and thus no disk writes occur. * PIM: handle SuspendPeer() before and after transfer ([FDO #82863](https://bugs.freedesktop.org/show_bug.cgi?id=82863)) A SuspendPeer() only succeeded while the underlying Bluetooth transfer was active. Outside of that, Bluez errors caused SyncEvolution to attempt a cancelation of the transfer and stopped the sync. When the transfer was still queueing, obexd returns org.bluez.obex.Error.NotInProgress. This is difficult to handle for SyncEvolution: it cannot prevent the transfer from starting and has to let it become active before it can suspend the transfer. Canceling would lead to difficult to handle error cases (like partially parsed data) and therefore is not done. The Bluez team was asked to implement suspending of queued transfers (see “org.bluez.obex.Transfer1 Suspend/Resume in queued state” on linux-bluetooth@vger.kernel.org), so this case might not happen anymore with future Bluez. When the transfer completes before obexd processes the Suspend(), org.freedesktop.DBus.Error.UnknownObject gets returned by obexd. SyncEvolution can ignore errors which occur after the active transfer completed. In addition, it should prevent starting the next one. This may be relevant for transfer in chunks, although the sync engine will also stop asking for data and thus typically no new transfer gets triggered anyway. * PIM: add suspend/resume/abort to sync.py CTRL-C while waiting for the end of a sync causes an interactive prompt to appear where one can choose been suspend/resume/abort and continuing to wait. CTRL-C again in the prompt aborts the script. * PIM: fix sync.py –sync-flags The help text used single quotes for the JSON example instead of the required double quotes. Running without –sync-flags was broken because of trying to parse the empty string as JSON. * command line: revise usability checking of datastores When configuring a new sync config, the command line checks whether a datastore is usable before enabling it. If no datastores were listed explicitly, only the usable ones get enabled. If unusable datastores were explicitly listed, the entire configure operation fails. This check was based on listing databases, which turned out to be too unspecific for the WebDAV backend: when “database” was set to some URL which is good enough to list databases, but not a database URL itself, the sources where configured with that bad URL. Now a new SyncSource::isUsable() operation is used, which by default just falls back to calling the existing Operations::m_isEmpty. In practice, all sources either check their config in open() or the m_isEmpty operation, so the source is usable if no error is enountered. For WebDAV, the usability check is skipped because it would require contacting a remote server, which is both confusing (why does a local configure operation need the server?) and could fail even for valid configs (server temporarily down). The check was incomplete anyway because listing databases gave a fixed help text response when no credentials were given. For usability checking that should have resulted in “not usable” and didn’t. The output during the check was confusing: it always said “listing databases” without giving a reason why that was done. The intention was to give some feedback while a potentially expensive operation ran. Now the isUsable() method itself prints “checking usability” if (and only if!) such a check is really done. Sometimes datastores were checked even when they were about to be configure as “disabled” already. Now checking such datastores is skipped. * EDS: memo syncing as iCalendar 2.0 ([FDO #52714](https://bugs.freedesktop.org/show_bug.cgi?id=52714)) When syncing memos with a peer which also supports iCalendar 2.0 as data format, the engine will now pick iCalendar 2.0 instead of converting to/from plain text. The advantage is that some additional properties like start date and categories can also be synchronized. The code is a lot simpler, too, because the EDS specific iCalendar 2.0 <-> text conversion code can be removed. * datatypes: text/calendar+plain revised heuristic When sending a VEVENT, DESCRIPTION was set to the SUMMARY if empty. This may have been necessary for some peers, but for notes (= VJOURNAL) we don’t know that (hasn’t been used in the past) and don’t want to alter the item unnecessarily, so skip that part and allow empty DESCRIPTION. When receiving a plain text note, the “text/calendar+plain” type used to store the first line as summary and the rest as description. This may be correct in some cases and wrong in others. The EDS backend implemented a different heuristic: there the first line is copied into the summary and stays in the description. This makes a bit more sense (the description alone is always enough to understand the note). Therefore and to avoid behavioral changes for EDS users when switching the EDS backend to use text/calendar+plain, the engine now uses the same approach. * source -> datastore rename, improved terminology The word “source” implies reading, while in fact access is read/write. “datastore” avoids that misconception. Writing it in one word emphasizes that it is single entity. While renaming, also remove references to explicit –*-property parameters. The only necessary use today is “–sync-property ?” and “–datastore-property ?”. –datastore-property was used instead of the short –store-property because “store” might be mistaken for the verb. It doesn’t matter that it is longer because it doesn’t get typed often. –source-property must remain valid for backward compatility. As many user-visible instances of “source” as possible got replaced in text strings by the newer term “datastore”. Debug messages were left unchanged unless some regex happened to match it. The source code will continue to use the old variable and class names based on “source”. Various documentation enhancements: Better explain what local sync is and how it involves two sync configs. “originating config” gets introduces instead of just “sync config”. Better explain the relationship between contexts, sync configs, and source configs (“a sync config can use the datastore configs in the same context”). An entire section on config properties in the terminology section. “item” added (Todd Wilson correctly pointed out that it was missing). Less focus on conflict resolution, as suggested by Graham Cobb. Fix examples that became invalid when fixing the password storage/lookup mechanism for GNOME keyring in 1.4. The “command line conventions”, “Synchronization beyond SyncML” and “CalDAV and CardDAV” sections were updated. It’s possible that the other sections also contain slightly incorrect usage of the terminology or are simply out-dated. * local sync: allow config name in syncURL=local:// Previously, only syncURL=local://@ was allowed and used the “target-config@context name” config as target side in the local sync. Now “local://config-name@context-name” or simply “local://config-name” are also allowed. “target-config” is still the fallback if only a context is give. It also has one more special meaning: “–configure target-config@google” will pick the “Google” template automatically because it knows that the intention is to configure the target side of a local sync. It does not know that when using some other name for the config, in which case the template (if needed) must be specified explicitly. The process name in output from the target side now also includes the configuration name if it is not the default “target-config”. * Google: remove SyncML template, combine CalDAV/CardDAV Google has turned off their SyncML server, so the corresponding “Google Contacts” template became useless and needs to be removed. It gets replaced by a “Google” template which combines the three different URLs currently used by Google for CalDAV/CardDAV. This new template can be used to configure a “target-config@google” with default calendar and address book database already enabled. The actual URL of these databases will be determined during the first sync using them. The template relies on the WebDAV backend’s new capability to search multiple different entries in the syncURL property for databases. To avoid listing each calendar twice (once for the legacy URL, once with the new one) when using basic username/password authentication, the backend needs a special case for Google and detect that the legacy URL does not need to be checked. * config: allow storing credentials for email address When configuring a WebDAV server with username = email address and no URL (which is possible if the server supports service discovery via the domain in the email address), then storing the credentials in the GNOME keyring used to fail with “cannot store password in GNOME keyring, not enough attributes”. That is because GNOME keyring seemed to get confused when a network login has no server name and some extra safeguards were added to SyncEvolution to avoid this. To store the credentials in the case above, the email address now gets split into user and domain part and together get used to look up the password. Upgrading from releases <= 1.3.99.4: ———————————— If the value of “username/databaseUser/proxyUser” contains a colon, the “user:” prefix must be added to the value, to continue treating it like a plain user name and not some reference to an unknown identity provider (like “id:”, “goa:”, “signon:”, etc.). The lookup of passwords in GNOME Keyring was updated slightly in 1.3.99.5. It may be necessary to set passwords anew if the old one is no longer found. Upgrading from release 1.2.x: —————————– The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts): syncevolution –configure \ syncFormat=text/x-vcard \ mobical addressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually: syncevolution –configure \ enableRefreshSync=TRUE \ funambol Upgrading from releases before 1.2: ———————————– Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Source, Installation, Further information ========================================= Source code bundles for users are available in http://downloads.syncevolution.org/syncevolution/sources and the original source is in [the git repositories](http://cgit.freedesktop.org/SyncEvolution/). i386, lpia and amd64 binaries for Debian-based distributions are available via the “unstable” syncevolution.org repository. Add the following entry to your /apt/source.list:
debhttp://downloads.syncevolution.org/aptunstablemain Then install “syncevolution-evolution”, “syncevolution-kde” and/or “syncevolution-activesync”. These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 10.04 LTS (Lucid), except for ActiveSync binaries which were compiled for Debian Wheezy, Ubuntu Saucy and Ubuntu Trusty. A backend for Ubuntu Online Accounts was compiled on Ubuntu Saucy. The packages mentioned above are meta-packages which pull in suitable packages matching the distro during installation. Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/). In contrast to 0.8.x archives, the 1.x .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps. More specific [HOWTOs](/wiki/howto) can be found in the Wiki.
This release enhances CalDAV/CardDAV and PBAP syncing and fixes some problems. The enhanced conflict handling introduced 1.4.99.2 was unintentionally limited to syncs with EDS on the server side; it is now also available for example in WebDAV<->SyncML bridge setups. Details: * CardDAV: implement read-ahead Instead of downloading contacts one-by-one with GET, SyncEvolution now looks at contacts that are most likely going to be needed soon and gets all of them at once with addressbook-multiget REPORT. The number of contacts per REPORT is 50 by default, configurable by setting the SYNCEVOLUTION_CARDDAV_BATCH_SIZE env variable. This has two advantages: - It avoids round-trips to the server and thus speeds up a large download (100 small contacts with individual GETs took 28s on a fast connection, 3s with two REPORTs). - It reduces the overall number of requests. Google CardDAV is known to start issuing intermittent 401 authentication errors when the number of contacts retrieved via GET is too large. Perhaps this can be avoided with addressbook-multiget. * Google Calendar: remove child hack, improve alarm hack ([FDO #63881](https://bugs.freedesktop.org/show_bug.cgi?id=63881)) Google recently enhanced support for RECURRENCE-ID, so SyncEvolution no longer needs to replace the property when uploading a single detached event with RECURRENCE-ID. However, several things are still broken in the server, with no workaround in SyncEvolution: - Removing individual events gets ignored by the server; a full “wipe out server data” might work (untested). - When updating the parent event, all child events also get updated even though they were included unchanged in the data sent by SyncEvolution. - The RECURRENCE-ID of a child event of an all-day recurring event does not get stored properly. - The update hack seems to fail for complex meetings: uploading them once and then deleting them seems to make uploading them again impossible. All of these issues were reported to Google and are worked on there, so perhaps the situation will improve. In the meantime, syncing with Google CalDAV should better be limited to: - Downloading a Google calendar in one-way mode. - Two-way syncing of simple calendars without complex meeting serieses. While updating the Google workarounds, the alarm hack (sending a new event without alarms twice to avoid the automatic server side alarm) was simplified. Now the new event gets sent only once with a pseudo-alarm. * ephemeral sync: don’t write binfile client files ([FDO #55921](https://bugs.freedesktop.org/show_bug.cgi?id=55921)) When doing PBAP caching, we don’t want any meta data written because the next sync would not use it anyway. With the latest libsynthesis we can configure “/dev/null” as datadir for the client’s binfiles and libsynthesis will avoid writing them. The PIM manager uses this for PBAP syncing automatically. For testing it can be enabled by setting the SYNCEVOLUTION_EPHEMERAL env variable. * PBAP: avoid empty field filter Empty field filter is supposed to mean “return all supported fields”. This used to work and stopped working with Android phones after an update to 4.3 (seen on Galaxy S3); now the phone only returns the mandatory TEL, FN, N fields. The workaround is to replace the empty filter list with the list of known and supported properties. This means we only pull data we really need, but it also means we won’t get to see any additional properties that the phone might support. * PBAP: transfer in chunks ([FDO #77272](https://bugs.freedesktop.org/show_bug.cgi?id=77272)) If enabled via env variables, PullAll transfers will be limited to a certain numbers contacts at different offsets until all data got pulled. See PBAP README for details. When transfering in chunks, the enumeration of contacts for the engine no longer matches the PBAP enumeration. Debug output uses “offset #x” for PBAP and “ID y” for the engine. * PBAP: remove transfer via pipe Using a pipe was never fully supported by obexd (blocks obexd). Transfering in suitably sized chunks ([FDO #77272](https://bugs.freedesktop.org/show_bug.cgi?id=77272)) will be a more obexd friendly solution with a similar effect (not having to buffer the entire address book in memory). * engine: enable batching by default ([FDO #52669](https://bugs.freedesktop.org/show_bug.cgi?id=52669)) This reverts commit c435e937cd406e904c437eec51a32a6ec6163102. Commit 7b636720a in libsynthesis fixes an unitialized memory read in the asynchronous item update code path. Testing confirms that we can now used batched writes reliably with EDS (the only backend currently supporting asynchronous writes + batching), so this change enables it again also for local and SyncEvolution<->SyncEvolution sync (with asynchronous execution of contact add/update overlapped with SyncML message exchanges) and other SyncML syncs (with changes combined into batches and executed at the end of each message). * datatypes: fix contact caching Adding grouping to the contact datatype in 1.4.99.2 broke PBAP caching: when sending an empty URL, for example, during the sync, the parsed contact had different field arrays than the locally stored contact, because the latter was saved without the empty URL. This caused the field-based comparison to detect a difference even when the final, reencoded contact wasn’t different at all. To solve this, syncing now uses the same “don’t send empty properties” configuration as local storages. Testing shows that this resolves the difference for EDS. A more resilient solution would be to add a check based on the encoded data, but that’s more costly performance wise. * datatypes: fix vCard handling The new “preserve repeating properties during conflict resolution” feature from 1.4.99.2 was only active when using EDS as storage. The relevant merge script must be applied to all datatypes, not just the EDS flavor. The feature was also unintentionally active when running in caching mode. This caused two problems: - The cached item was updated even though only the ordering of repeating properties had been modified during merging. - The merged item was sent back to the client side, which was undesirable (caching is supposed to be one-way) or even impossible (PBAP is read-only, causing sync failures eith error 20030). We must check for caching mode and disable merging when it is active. We also must not tell the engine that we updated the photo property in the winning item, because then that item would get sent to the read-only side of the sync. Perhaps a better solution would be to actually tell the engine that the remote side is read-only when we activate caching mode. * datatypes: avoid PHOTO corruption during merge ([FDO #77065](https://bugs.freedesktop.org/show_bug.cgi?id=77065)) When handling an update/update conflict (both sides of the sync have an updated contact) and photo data was moved into a local file by EDS, the engine merged the file path and the photo data together and thus corrupted the photo. The engine does not know about the special role of the photo property. This needs to be handled by the merge script, and that script did not cover this particular situation. Now the loosing side is cleared, causing the engine to then copy the winning side over into the loosing one. Found by Renato Filho/Canonical when testing SyncEvolution for Ubuntu 14.04. * PBAP syncing: updated photo not always stored Because photo data was treated like a C string, changes after any embedded null byte were ignored during a comparison. * PIM: fix phone number normalization The parsed number always has a country code, whereas SyncEvolution expected it to be zero for strings without an explicit country code. This caused a caller ID lookup of numbers like “089788899” in DE to find only telephone numbers in the current default country, instead of being more permissive and also finding “+189788899”. The corresponding unit test was broken and checked for the wrong result. Found while investigating an unrelated test failure when updating libphonenumber. * Various compiler problems and warnings fixed; compiles with –with-warnings=fatal on current Debian Testing and Ubuntu Trusty ([FDO #79316](https://bugs.freedesktop.org/show_bug.cgi?id=79316)). Upgrading from releases <= 1.3.99.4: ———————————— If the value of “username/databaseUser/proxyUser” contains a colon, the “user:” prefix must be added to the value, to continue treating it like a plain user name and not some reference to an unknown identity provider (like “id:”, “goa:”, “signon:”, etc.). The lookup of passwords in GNOME Keyring was updated slightly in 1.3.99.5. It may be necessary to set passwords anew if the old one is no longer found. Upgrading from release 1.2.x: —————————– The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts): syncevolution –configure \ syncFormat=text/x-vcard \ mobical addressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually: syncevolution –configure \ enableRefreshSync=TRUE \ funambol Upgrading from releases before 1.2: ———————————– Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Source, Installation, Further information ========================================= Source code bundles for users are available in http://downloads.syncevolution.org/syncevolution/sources and the original source is in [the git repositories](http://cgit.freedesktop.org/SyncEvolution/). i386, lpia and amd64 binaries for Debian-based distributions are available via the “unstable” syncevolution.org repository. Add the following entry to your /apt/source.list:
debhttp://downloads.syncevolution.org/aptunstablemain Then install “syncevolution-evolution”, “syncevolution-kde” and/or “syncevolution-activesync”. These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 10.04 LTS (Lucid), except for ActiveSync binaries which were compiled for Debian Wheezy, Ubuntu Saucy and Ubuntu Trusty. A backend for Ubuntu Online Accounts was compiled on Ubuntu Saucy. The packages mentioned above are meta-packages which pull in suitable packages matching the distro during installation. Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/). In contrast to 0.8.x archives, the 1.x .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps. More specific [HOWTOs](/wiki/howto) can be found in the Wiki.
Here’s a copy of the slides that I presented at the Tizen Developer Conference 2014 in San Francisco. Compared to the previous talk at the Automotive Linux Summit in Edinburgh, you’ll find Tizen-specific instructions for getting started and information about the new OAuth2 (or, as a fallback, password based) Google Contact sync. *[Update]*: a copy of the slides and an audio recording of the talk are now also on the [conference web site](https://www.tizen.org/events/tizen-developer-conference/2014/conference-schedule) (search for “PIM in IVI - Contacts, your phone, your car”).
1.4.99.2 is a new development snapshot. It enhances interoperability with CardDAV servers and in particular Google Contacts considerably. Contact data gets converted to and from the format typically used by CardDAV servers, so now anniversary, spouse, manager, assistant and instant message information are exchanged properly. Categories are not supported by Google CardDAV and thus still get lost. Custom labels get stored in EDS as extensions and no longer get lost when updating some other aspects of a contact. However, Evolution does not show custom labels and removes them when editing a property which has a custom label (BGO #730636). Scanning for CardDAV/CalDAV resources was enhanced. It now finds additional calendars with Google CalDAV and works with iCloud. However, syncing with iCloud ran into a server bug (reported as 17001498 “CalDAV REPORT drops calendar data”) and needs further work. The Ubuntu Online Accounts backend was added to syncevolution.org binaries, targeting Ubuntu Saucy and later. Details: * vcard profile: avoid data loss during merging When resolving a merge conflict, repeating properties were taken wholesale from the winning side (for example, all email addresses). If a new email address had been added on the loosing side, it got lost. Arguably it is better to preserve as much data as possible during a conflict. SyncEvolution now does that in a merge script by checking which properties in the loosing side do not exist in the winning side and copying those entries. Typically only the main value (email address, phone number) is checked and not the additional meta data (like the type). Otherwise minor differences (for example, both sides have same email address, but with different types) would lead to duplicates. Only addresses are treated differently: for them all attributes (street, country, city, etc.) are compared, because there is no single main value. * engine: UID support in contact data Before, the UID property in a vCard was ignored by the engine. Backends were responsible for ensuring that the property is set if required by the underlying storage. This turned out to be handled incompletely in the WebDAV backend. This change moves this into the engine: - UID is now field. It does not get used for matching because the engine cannot rely on it being stored by both sides. - It gets parsed if present, but only generated if explicitly enabled (because that is the traditional behavior). - It is never shown in the DevInf’s CtCap because the Synthesis engine would always show it regardless whether a rule enabled the property. That’s because rules normally only get triggered after exchanging DevInf and thus DevInf has to be rule-independent. We don’t want it shown because then merging the incoming item during a local sync would use the incoming UID, even if it is empty. - Before writing, ensure that UID is set. When updating an existing item, the Synthesis engine reads the existing item, preserves the existing UID unless the peer claims to support UID, and then updates with the existing UID. This works for local sync (where SyncEvolution never claims to support UID when talking to the other side). It will break with peers which have UID in their CtCap although they rewrite the UID and backends whose underlying storage cannot handle UID changes during an update (for example, CardDAV). * CardDAV: use Apple/Google/CardDAV vCard flavor In principle, CardDAV servers support arbitrary vCard 3.0 data. Extensions can be different and need to be preserved. However, when multiple different clients or the server’s Web UI interpret the vCards, they need to agree on the semantic of these vCard extensions. In practice, CardDAV was pushed by Apple and Apple clients are probably the most common clients of CardDAV services. When the Google Contacts Web UI creates or edits a contact, Google CardDAV will send that data using the vCard flavor used by Apple. Therefore it makes sense to exchange contacts with *all* CardDAV servers using that format. This format could be made configurable in SyncEvolution on a case-by-case basis; at the moment, it is hard-coded. During syncing, SyncEvolution takes care to translate between the vCard flavor used internally (based on Evolution) and the CardDAV vCard flavor. This mapping includes: X-AIM/JABBER/… <-> IMPP + X-SERVICE-TYPE Any IMPP property declared as X-SERVICE-TYPE=AIM will get mapped to X-AIM. Same for others. Some IMPP service types have no known X- property extension; they are stored in EDS as IMPP. X- property extensions without a known X-SERVICE-TYPE (for example, GaduGadu and Groupwise) are stored with X-SERVICE-TYPE values chosen by SyncEvolution so that Google CardDAV preserves them (GroupWise with mixed case got translated by Google into Groupwise, so the latter is used). Google always sends an X-ABLabel:Other for IMPP. This is ignored because the service type overrides it. The value itself also gets transformed during the mapping. IMPP uses an URI as value, with a chat protocol (like “aim” or “xmpp”) and some protocol specific identifier. For each X- extension the protocol is determined by the property name and the value is the protocol specific identifier without URL encoding. X-SPOUSE/MANAGER/ASSISTANT <-> X-ABRELATEDNAMES + X-ABLabel The mapping is based on the X-ABLabel property attached to the X-ABRELATEDNAMES property. This depends on the English words “Spouse”, “Manager”, “Assistant” that Google CardDAV and Apple devices seem to use regardless of the configured language. As with IMPP, only the subset of related names which have a corresponding X- property extension get mapped. The rest is stored in EDS using the X-ABRELATEDNAMES property. X-ANNIVERSARY <-> X-ABDATE Same here, with X-ABLabel:Anniversary as the special case which gets mapped. X-ABLabel parameter <-> property CardDAV vCards have labels attached to arbitrary other properties (TEL, ADR, X-ABDATE, X-ABRELATEDNAMES, …) via vCard group tags: item1.X-ABDATE:2010-01-01 item1.X-ABLabel:Anniversary The advantage is that property values can contain arbitrary characters, including line breaks and double quotation marks, which is not possible in property parameters. Neither EDS nor KDE (judging from the lack of responses on the KDE-PIM mailing list) support custom labels. SyncEvolution could have used grouping as it is done in CardDAV, but grouping is not used much (not at all?) by the UIs working with the vCards in EDS and KDE. It seemed easier to use a new X-ABLabel parameter. Characters which cannot be stored in a parameter get converted (double space to single space, line break to space, etc.) during syncing. In practice, these characters don’t appear in X-ABLabel properties anyway because neither Apple nor Google UIs allow entering them for custom labels. The “Other” label is used by Google even in case where it adds no information. For example, all XMPP properties have an associated X-ABLabel=Other although the Web UI does not provide a means to edit or show such a label. Editing the text before the value in the UI changes the X-SERVICE-TYPE parameter value, not the X-ABLabel as for other fields. Therefore the “Other” label is ignored by removing it during syncing. X-EVOLUTION-UI-SLOT (the parameter used in Evolution to determine the order of properties in the UI) gets stored in CardDAV. The only exception is Google CardDAV which got confused when an IMPP property had both X-SERVICE-TYPE and X-EVOLUTION-UI-SLOT parameters set. For Google, X-EVOLUTION-UI-SLOT is only sent on other properties and thus ordering of chat information can get lost when syncing with Google. * synccompare: support grouping and quoted parameter strings Grouped properties are sorted first according to the actual property name, then related properties are moved to the place where their group tag appears first. The first grouped property gets a “- ” prefix, all following ones are just indended with ” “. The actual group tag is not part of the normalized output, because its value is irrelevant: BDAY:19701230 - EMAIL:john@custom.com X-ABLabel:custom-label2 … FN:Mr. John 1 Doe Sr. - IMPP;X-SERVICE-TYPE=AIM:aim:aim X-ABLabel:Other … - X-ABDATE:19710101 X-ABLabel:Anniversary Redundant tags (those set for only a single property, X-ABLabel:Other) get removed as part of normalizing an item. * WebDAV: use server’s order when listing collections When doing a recursive scan of the home set, preserve the order of entries as reported by the server and check the first one first. The server knows better which entries are more relevant for the user (and thus should be the default) or may have some other relevant order. Previously, SyncEvolution replaced that order with sorting by URL, which led to a predictable, but rather meaningless order. For example, Google lists the users own calendar first, followed by the shared calendars sorted alphabetical by their name. Now SyncEvolution picks the main calendar as default correctly when scanning from https://www.google.com/calendar/dav/. * WebDAV: improved database search (Google, Zimbra) Zimbra has a principal URL that also serves as home set. When using it as start URL, SyncEvolution only looked the URL once, without listing its content, and thus did not find the databases. When following the Zimbra principal URL indirectly, SyncEvolution did check all of the collections there recursively. Unfortunately that also includes many mail folders, causing the scan to abort after checking 1000 collections (an internal safe guard). The solution for both includes tracking what to do with a URL. For the initial URL, only meta data about the URL itself gets checked. Recursive scanning is only done for the home set. If that home set contains many collections, scanning is still slow and may run into the internal safe guard limit. This cannot be avoided because the CalDAV spec explicitly states that the home set may contain normal collections which contain other collections, so a client has to do the recursive scan. When looking at a specific calendar, Google CalDAV does not report what the current principal or the home set is and therefore SyncEvolution stopped after finding just the initial calendar. Now it detects the lack of meta information and adds all parents also as candidates that need to be looked at. The downside of this is that it doesn’t know anything about which parents are relevant, so it ends up checking https://www.google.com/calendar/ and https://www.google.com/. In both cases Basic Auth gets rejected with a temporary redirect to the Google login page, which is something that SyncEvolution must ignore immediately during scanning without applying the resend workaround for “temporary rejection of valid credentials” that can happen for valid Google CalDAV URLs. * WebDAV: enhanced database search (Google Calendar) Additional databases where not found for several reasons. SyncEvolution ignored all shared calendars (http://calendarserver.org/ns/shared) and Google marks the additional calendars that way. The other problem was that the check for leaf collections (= collections which cannot contain other desired collections) incorrectly excluded those collections instead of only preventing listing of their content. With this change, https://www.google.com/calendar/dav/?SyncEvolution=Google can be used as starting point for Google Calendar. * WebDAV: fix database scan on iCloud The calendar home set URL on iCloud (the one ending in /calendars/) is declared as containing calendar data. That was enough for SyncEvolution to accept it incorrectly as calendar. However, the home set only contains calendar data indirectly. * WebDAV: support redirects between hosts and DNS SRV lookup based on URL When finding a new URL, we must be prepared to reinitialize the Neon session with the new host settings. iCloud does not have .well-known support on its www.icloud.com server. To support lookup with a non-icloudd.com email address, we must do DNS SRV lookup when access to .well-known URLs fails. We do this without a www prefix on the host first, because that is what happens to work for icloud.com. With these changes it becomes possible to do database scans on Apple iCloud, using syncURL=https://www.icloud.com or syncURL=https://icloud.com. Giving the syncURL like this is only necessary for a username that does not end in @icloud.com. When the syncURL is not set, the domain for DNS SRV lookup is taken from the username. * WebDAV: more efficient item creation PUT has the disadvantage that a client needs to choose a name and then figure out what the real name on the server is. With Google CardDAV that requires sending another request and only works because the server happens to remember the original name (which is not guaranteed!). POST works for new items without a name and happens to be implemented by Google such that the response already includes all required information (new name and revision string). POST is checked for as described in RFC 5995 once before creating a new item. Servers which don’t support it continue to get a PUT. * WebDAV: send “User-Agent: SyncEvolution” Apple iCloud servers reject requests unless they contain a User-Agent header. The exact value doesn’t seem to matter. Making the string configurable might be better, but can still be done later when it is more certain whether and for what it is needed. * WebDAV: refactor and fix DNS SRV lookup The syncevo-webdav-lookup script was not packaged. It did not report “not found” DNS results correctly and the caller did not check for this either, so when looking up the information for a domain which does not have DNS SRV entries, SyncEvolution ended up retrying for while as if there had been a temporary lookup problem. * signon: make Accounts optional The new “signon” provider only depends on lib[g]signon-glib. It uses gSSO if found, else UOA. Instead of pulling parameters and the identity via libaccounts-glib, the user of SyncEvolution now has to ensure that the identity exists and pass all relevant parameters in the “signon:” username. * gSSO: adapt to gSSO >= 2.0 * config templates: Funambol URLs Funambol turned of the URL redirect from my.funambol.com to onemedia.com. The Funambol template now uses the current URL. Users with existing Funambol configs must updated the syncURL property manually to https://onemediahub.com/sync Kudos to Daniel Clement for reporting the change. * command line: fix –update from directory The “–update
The first bug fix release in the 1.4 series addresses some issues which occurred on some systems. Several issues with Akonadi were fixed. Details: ——— * EDS: only load one backend plugin of each kind SyncEvolution was meant to load the syncecal or syncebook shared object which uses the most recent libraries (libical, libecal/libebook) on the system and then stop loooking for alternatives. Due to a string handling bug the check for already backends always found nothing, leading to multiple conflicting backends loaded on some systems (for example, those with libical0 and libical1 installed). If that happened, the backend became unusable. * ical: workaround for libical 1.0 builtin timezone change libical 1.0 started to return VTIMEZONE definitions with multiple absolute transition times instead of RRULEs. This causes problems when exchanging data with peers (see https://sourceforge.net/p/freeassociation/bugs/95/). In SyncEvolution, this affected sending an event using New Zealand time in vCalendar 1.0 format to a phone, because the internal, out-dated definition of the time zone in libsynthesis was used as fallback when loading RRULE-based timezone definitions from libical failed (see “[SyncEvolution] Some events showing wrong time on phone”). It might also affect exchanging data with CalDAV peers (not tested). The workaround is to include the original code from libical. * dbus-session.sh: create XDG_RUNTIME_DIR More recent distros (for example, Ubuntu Saucy) rely on XDG_RUNTIME_DIR. Each time dbus-session.sh runs, it must ensure that the runtime dir exists and is empty. This was a problem when trying to run activesyncd + SyncEvolution on a headless Ubuntu Saucy server (see [FDO #76273](https://bugs.freedesktop.org/show_bug.cgi?id=76273)). * Akonadi: support KDE Notes, enhanced “database” check The KDE Notes resources store items under a different MIME type than the one used in AKonadi (see “[Kde-pim] note format”). SyncEvolution use the same type as Akonadi and thus did not find existing KDE Notes resources. To support both while KDE and Akonadi transition to the same type, SyncEvolution now looks for notes resources using both MIME types and accepts both kinds of items when reading. When writing, SyncEvolution picks the MIME type that is supported by the resource, which hopefully avoids confusing the KDE app using the resource (untested). As a positive side effect, the “database” value used for opening a resource is now checked more thoroughly. Non-existent resources and the type mismatches like pointing a “kde-contacts” backend to a calendar resource are now detected early. * Akonadi: ensure that UID is set ([FDO #74342](https://bugs.freedesktop.org/show_bug.cgi?id=74342)) Akonadi resources do not enforce iCalendar 2.0 semantic like “each VEVENT must have a UID” (see “[Kde-pim] iCalendar semantic”). When receiving an event from a peer which itself does not enforce that semantic (Funambol, vCalendar 1.0 based phones), then we need to generate a UID, otherwise KOrganizer will ignore the imported event. * Akonadi: avoid threading problem in HTTP server mode ([FDO #75672](https://bugs.freedesktop.org/show_bug.cgi?id=75672)) When used as storage in a server, Akonadi got called in a background thread that gets created to handle slow initialization of sources and preventing ensuing timeouts in HTTP clients (probably not needed for Akonadi itself, but may still be useful when combining it with other sources). Akonadi cannot be used like that, leading to false “Akonadi not running” errors or (if one got past that check) failing item operations. * autotools: Add QtCore include path to KDEPIM_CFLAGS ([FDO #75670](https://bugs.freedesktop.org/show_bug.cgi?id=75670)) This fixes an issue where configure fails to find Akonadi when test programs do not compile because QString is not found. * Enhanced testing again: faster execution, less false negatives under load. Re-enabled testing of Akonadi. Upgrading from releases <= 1.3.99.4: ———————————— If the value of “username/databaseUser/proxyUser” contains a colon, the “user:” prefix must be added to the value, to continue treating it like a plain user name and not some reference to an unknown identity provider (like “id:”, “goa:”, “signon:”, etc.). The lookup of passwords in GNOME Keyring was updated slightly in 1.3.99.5. It may be necessary to set passwords anew if the old one is no longer found. Upgrading from release 1.2.x: —————————– The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts): syncevolution –configure \ syncFormat=text/x-vcard \ mobical addressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually: syncevolution –configure \ enableRefreshSync=TRUE \ funambol Upgrading from releases before 1.2: ———————————– Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Source, Installation, Further information ========================================= Source code bundles for users are available in http://downloads.syncevolution.org/syncevolution/sources and the original source is in [the git repositories](http://cgit.freedesktop.org/SyncEvolution/). i386, lpia and amd64 binaries for Debian-based distributions are available via the “stable” syncevolution.org repository. Add the following entry to your /apt/source.list:
debhttp://downloads.syncevolution.org/aptstablemain Then install “syncevolution-evolution”, “syncevolution-kde” and/or “syncevolution-activesync”. These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 10.04 LTS (Lucid), except for ActiveSync binaries which were compiled for Debian Wheezy, Ubuntu Saucy and Ubuntu Trusty. The packages mentioned above are meta-packages which pull in suitable packages matching the distro during installation. Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/). In contrast to 0.8.x archives, the 1.x .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps. More specific [HOWTOs](/wiki/howto) can be found in the Wiki.
The 1.4 release of SyncEvolution replaces 1.3.2 as the stable, supported release. 1.4 is the first stable version with the in-vehicle infotainment (IVI) PIM Manager included. GENIVI Diagnostic Log and Trace (DLT) is also supported. For more information about this aspect of SyncEvolution, see the PBAP and PIM entries in the [1.3.99 release notes](http://cgit.freedesktop.org/SyncEvolution/syncevolution/tree/NEWS?id=syncevolution-1-3-99-7) and these [Automotive Linux Summit slides](https://syncevolution.org/blogs/pohly/2013/pim-its-all-about-contacts). The biggest change for normal Linux users is Google CalDAV/CardDAV authentication with OAuth2. These are the open protocol that Google currently supports and thus the recommended way of syncing with Google, replacing ActiveSync and SyncML (both no longer available to all Google customers). Support for Google CardDAV is new. Like Evolution, SyncEvolution does not yet support some of the advanced features of the server, in particular custom labels for phone numbers, emails and addresses. Likewise, some client properties are not supported by the server: CALURI, CATEGORIES, FBURL, GEO and ROLE are not supported. Of ORG, only the first two components are supported. Currently, properties not supported by one side get lost in a full roundtrip sync. Instant Messaging information is supported by both sides with different vCard extensions; the server stores these extensions without showing the information, while SyncEvolution drops the data sent by the server. SyncEvolution depends on external components for OAuth2. It can be compiled to use [gSSO](https://01.org/gsso) or GNOME Online Accounts. The latter is enabled in binaries from syncevolution.org. GNOME Online Accounts >= 3.10 works out of the box for CalDAV and CardDAV. 3.8 is guaranteed to work for CalDAV and may also work for CardDAV, if the Linux distribution ships a patched version (like Debian Testing/Jessie does). If it does not, then GNOME Online Accounts 3.8 binary can be patched to also support CardDAV, see the GOA README. Anything older than 3.8 does not work. Support for Ubuntu Online Accounts is available when compiling from source. For setup instructions see the [GOA](http://cgit.freedesktop.org/SyncEvolution/syncevolution/plain/src/backends/goa/README) and [signon](http://cgit.freedesktop.org/SyncEvolution/syncevolution/plain/src/backends/signon/README) READMEs. Binary packages of 1.4 on syncevolution.org have enhanced support for recent distros. They now work with EDS >= 3.6 *and* < 3.6. Distros with libical1 like Ubuntu Saucy are also supported. The HTTP server became better at handling message resends when the server is slow with processing a message. The server is able to keep a sync session alive while loading the initial data set by sending acknowledgment replies before the client times out. Some issues in CalDAV, WebDAV and SyncML were fixed. Graham R. Cobb contributed several patches for enhancing ActiveSync support and making it work with Exchange 2010. Guido Günther provided some patches addressing problems when compiling SyncEvolution for Maemo. Details: ——– * D-Bus server: support DLT ([FDO #66769](https://bugs.freedesktop.org/show_bug.cgi?id=66769)) Diagnostic Log and Trace (DLT) manages a sequence of log messages, with remote controllable level of detail. SyncEvolution optionally (can be chosen at compile time and again at runtime) uses DLT instead of its own syncevolution-log.html files. See README-DLT.rst for more information. To use the feature, configure SyncEvolution with “–enable-dbus-server=–dlt –no-syslog” * D-Bus server: fix abort when mixing auto-sync and manual operations ([FDO #73562](https://bugs.freedesktop.org/show_bug.cgi?id=73562)) When enabling auto-sync for a config and then accessing or syncing the config manually via the command line tool, the server would abort at the time when the auto-sync was originally scheduled. * D-Bus server: accept WBXML with charset in incoming connections A user reported via email that the Nokia 515 sends ‘application/vnd.syncml+wbxml; charset=UTF-8’ as type of its messages this tripped up the syncevo-http-server, leading to: [ERROR] syncevo-dbus-server: /org/syncevolution/Server: message type ‘application/vnd.syncml+wbxml; charset=UTF-8’ not supported for starting a sync * D-Bus server: command line options for controlling output and startup The system log is used by default now. New command line options can be used to change this:
-d,--duration=seconds/'unlimited'Shutdownautomaticallywhenidleforthisduration(default300seconds)-v,--verbosity=levelChooseamountofoutput,0=nooutput,1=errors,2=info,3=debug;defaultis1.--dbus-verbosity=levelChooseamountofoutputviaD-Bussignals,0=nooutput,1=errors,2=info,3=debug;defaultis2.-o,--stdoutEnableprintingtostdout(resultofoperations)andstderr(errors/info/debug).-s,--no-syslogDisableprintingtosyslog.-p,--start-pimActivatethePIMManager(=unifiedaddressbook)immediately. * D-Bus: missing out parameters in D-Bus introspection XML ([FDO #57292](https://bugs.freedesktop.org/show_bug.cgi?id=57292)) The problem was in the C++ D-Bus binding. If the method that gets bound to D-Bus returns a value, that value was ignored in the signature: int foo() => no out parameter It works when the method was declared as having a retval: void foo (int &result) => integer out parameter This problem existed for both the libdbus and the GIO D-Bus bindings. In SyncEvolution it affected methods like GetVersions(). * D-Bus server: avoid progress outside of 0-100% range For example in the new TestLocalCache.testItemDelete100, the percentage value in the ProgressChanged signal become larger than 100 and then revert to 100 at the end of the sync. Seems the underlying calculation is faulty or simply inaccurate. This is not fixed. Instead the result is just clipped to the valid range. * sync: less verbose output, shorter runtime For each incoming change, one INFO line with “received x[/out of y]” was printed, immediately followed by another line with total counts “added x, updated y, removed z”. For each outgoing change, a “sent x[/out of y]” was printed. In addition, these changes were forwarded to the D-Bus server where a “percent complete” was calculated and broadcasted to clients. All of that caused a very high overhead for every single change, even if the actual logging was off. The syncevo-dbus-server was constantly consuming CPU time during a sync when it should have been mostly idle. To avoid this overhead, the updated received/sent numbers that come from the Synthesis engine are now cached and only processed when done with a SyncML message or some other event happens (whatever happens first). To keep the implementation simple, the “added x, updated y, removed z” information is ignored completely and no longer appears in the output. * command line: implement –create/remove-database Creating a database is only possible with a chosen name. The UID is chosen automatically by the storage. Only implemented in the EDS backend. * command line: execute –export and –print-items while the source is still reading Instead of reading all item IDs, then iterating over them, process each new ID as soon as it is available. With sources that support incremental reading (only the PBAP source at the moment) that provides output sooner and is a bit more memory efficient. * command line: recover from slow sync with new sync modes The error message for an unexpected slow sync still mentioned the old and obsolete “refresh-from-client/server” sync modes. Better mention “refresh-from-local/remote”. * command line: show backend error when listing databases fails The command line swallowed errors thrown by the backend while listing databases. Instead it just showed “: backend failed”. The goal was to not distract users who accidentally access a non-functional backend. But the result is that operations like –configure or –print-databases could fail without giving the user any hint about the root cause of the issue. Now the error explanation in all its gory details is included. For example, not having activesyncd running leads to: INFO] eas_contact: backend failed: fetching folder list: GDBus.Error:org.freedesktop.DBus.Error.ServiceUnknown: The name org.meego.activesyncd was not provided by any .service files And running activesyncd without the necessary gconf keys shows up as: [INFO] eas_contact: backend failed: fetching folder list: GDBus.Error:org.meego.activesyncd.Error.AccountNotFound: Failed to find account [syncevolution@lists.intel.com] * password handling: fix usage of GNOME Keyring and KWallet ([FDO #66110](https://bugs.freedesktop.org/show_bug.cgi?id=66110)) When clients like the GTK sync-ui stored a password, it was always stored as plain text in the config.ini file by the syncevo-dbus-server. The necessary code for redirecting the password storage in a keyring (GNOME or KWallet) simply wasn’t called in that case. The command line tool, even when using the D-Bus server to run the operation, had the necessary code active and thus was not affected. Now all SyncEvolution components use the same default: use safe password storage if either GNOME Keyring or KWallet were enabled during compilation, don’t use it if not. Fixing this revealed other problems, like not being able to store certain passwords that lacked the necessary lookup criteria (like syncURL and/or username). To address this, the lookup criteria where extended and a new check was added to avoid accidentally removing other passwords. As a result, it may be possible that SyncEvolution no longer finds passwords that were stored with older versions of SyncEvolution. In such a case the passwords must be set again. * GNOME: clean up keyring access and require libgnome-keyring >= 2.20 The updated error messages now always include information about the password and libgnome-keyring error texts. A workaround is used for the “Error communicating with gnome-keyring-daemon” problem that started to appear fairly frequently in the automated testing once the keyring was actually used. The problem shows up with some additional debug messages: Gkr: received an invalid, unencryptable, or non-utf8 secret Gkr: call to daemon returned an invalid response: (null).(null)() It seems that sometimes setting up a session with GNOME keyring fails such that all further communication leads to decoding problem. There is an internal method to reset the session, but it cannot be called directly. As a workaround, fake the death of the GNOME keyring daemon and thus trigger a reconnect when retrying the GNOME keyring access. This is done by sending a D-Bus message, which will also affect other clients of GNOME keyring, but hopefully without user-visible effects. * config: enhanced password handling It is possible to configure a plain username/password combination once in SyncEvolution and then use references to it in other configurations, instead of having to set (and update) the credentials in different places. This is useful in particular with WebDAV, where credentials had to be repeated several times (target config, in each database when used as part of SyncML) or when using a service which requires several configs (Google via SyncML and CalDAV). To use this, create a sync config for a normal peer or a dedicated config just for the credentials, with “username/password/syncURL” set. The “syncURL” must be set to something identifying the peer if GNOME Keyring is used for the password storage. Then set “username”, “databaseUser” and “proxyUser” properties to “id:” and all read and write access to those properties will be redirected by SyncEvolution into that other configuration. This even works in the GTK UI. For user names which contain colons, the new “user:” format must be used. Strings without colons are assumed to be normal user names, so most old configurations should continue to work. * signon: new backend using libgsignond-glib + libaccounts-glib The code works with gSSO (https://01.org/gsso) and Ubuntu Online Accounts. * GOA: get OAuth2 tokens out of GNOME Online Accounts “username = goa:…” selects an account in GOA and retrieves the OAuth2 token from that. * WebDAV: support OAuth2 If given an authentication configuration which can handle OAuth2, then OAuth2 is used instead of plain username/password authentication. * WebDAV: support Google CardDAV, break Yahoo Google CardDAV has one peculiarity: it renames new contacts during PUT without returning the new path to the client. See also http://lists.calconnect.org/pipermail/caldeveloper-l/2013-July/000524.html SyncEvolution already had a workaround for that (PROPGET on old path, extract new path from response) which happened to work. This workaround was originally added for Yahoo, which sometimes merges contacts into existing ones. In contrast to Yahoo, Google really seems to create new items. Without some server specific hacks, the client cannot tell what happened. Because Google is currently supported and Yahoo is not, let’s change the hard-coded behavior to “renamed items are new”. * WebDAV: started testing with owndrive.com = OwnCloud * WebDAV: avoid segfault during collection lookup Avoid referencing pathProps->second when the set of paths that PROPFINDs returns is empty. Apparently this can happen in combination with Calypso. * CalDAV: more workarounds for Google CalDAV + unique IDs Google became even more strict about checking REV. Tests which reused a UID after deleting the original item started to fail sometime since middle of December 2012. * CalDAV: work around Google server regression (undeclared namespace prefix in XML) Google CalDAV for a while (December 2012 till January 2013) sent invalid XML back when asked to include CardDAV properties in a PROPFIND. This got rejected in the XML parser, which prevents syncing calendar data: Neon error code 1: XML parse error at line 55: undeclared namespace prefix In the meantime Google fixed the issue in response to a bug report via email. But the workaround, only asking for the properties which are really needed, still makes sense and thus is kept. * WebDAV: auto-discovery fix With Google Contact + CardDAV the auto-discovery failed after finding the default address book, without reporting that result. * WebDAV: don’t send Basic Auth via http proactively ([FDO #57248](https://bugs.freedesktop.org/show_bug.cgi?id=57248)) Sending basic authentication headers via http is insecure. Only do it proactively when the connection is encrypted and thus protects the information or when the server explicitly asks for it. * file backend: sub-second mod time stamps Change tracking in the file backend used to be based on the modification time in seconds. When running many syncs quickly (as in testing), that can lead to changes not being detected when they happen within a second. Now the file backend also includes the sub-second part of the modification time stamp, if available. This change is relevant when upgrading SyncEvolution: most of the items will be considered “updated” once during the first sync after the upgrade (or a downgrade) because the revision strings get calculated differently. * GTK UI: fixed two crashes - running a sync with no service selected and a 64 bit pointer problem recently discovered by Tino Keitel when compiling the Debian package with -fPIE. * packaging: compatible with EDS up to and including 3.10 and both libical.so.0 and libical.so.1 The binary packages now contain different versions of syncecal.so and syncebooks.so to cover different combinations of EDS and libical. * libical: compatibiliy mode for libical.so.0 and libical.so.1 libical 1.0 broke the ABI, leading to libical.so.1. The only relevant change for SyncEvolution is the renumbering of ICAL_*_PROPERTY enum values. We can adapt to that change at runtime, which allows us to compile once with libical.so.0, then patch executables or use dynamic loading to run with the more recent libical.so.1 if we add 1 to the known constants. * packaging: fix rpm ([FDO #73347](https://bugs.freedesktop.org/show_bug.cgi?id=73347)) After installing the syncevolution.org rpm on OpenSUSE, SyncEvolution was not starting because its shared libraries were not found unless “ldconfig” was called manually. Now the package does that automatically. * packaging: fix description The syncevolution-bundle description of both rpm and deb packagesaccidentally used the same description as syncevolution-evolution. * glib: fix double-free of source tags glib 2.39.0 (aka GNOME 3.10) as found in Ubuntu Trusty introduces warnings when g_source_remove() is passed an unknown tag. SyncEvolution did this in two cases: in both, the source callback returned false and thus caused the source to be removed by the caller. In that case, the explicit g_source_remove() is redundant and must be avoided. Such a call is faulty and might accidentally remove a new source with the same tag (unlikely though, given that tags seem to get assigned incrementally). The only noticable effect were additional error messages with different numbers: [ERROR] GLib: Source ID 9 was not found when attempting to remove it * EDS: fix compile problem with boost and EDS > 3.36 This fixes the following problem, seen with Boost 1.53.0 on altlinux when compiling for EDS >= 3.6: /usr/include/boost/smart_ptr/shared_ptr.hpp: In instantiation of ‘typename boost::detail::sp_array_access::type boost::shared_ptr::operator const [with T = char*; typename boost::detail::sp_array_access::type = void; std::ptrdiff_t = long int]’: src/backends/evolution/EvolutionSyncSource.cpp:163:38: required from here /usr/include/boost/smart_ptr/shared_ptr.hpp:663:22: error: return-statement with a value, in function returning ‘void’ [-fpermissive] make[2]: *** [src/backends/evolution/src_backends_evolution_syncecal_la-EvolutionSyncSource.lo] * EDS contacts: avoid unnecessary DB writes during slow sync Traditionally, contacts were modified shortly before writing into EDS to match with Evolution expectations (must have N, only one CELL TEL, VOICE flag must be set). During a slow sync, the engine compare the modified contacts with the unmodified, incoming one. This led to mismatches and/or merge operations which end up not changing anything in the DB because the only difference would be removed again before writing. * EDS contacts: read-ahead cache Performance is improved by requesting multiple contacts at once and overlapping reading with processing. On a fast system (SSD, CPU fast enough to not be the limiting factor), testpim.py’s testSync takes 8 seconds for a “match” sync where 1000 contacts get loaded and compared against the same set of contacts. Read-ahead with only 1 contact per query speeds that up to 6.7s due to overlapping IO and processing. Read-ahead with the default 50 contacts per query takes 5.5s. It does not get much faster with larger queries. * PBAP: add support for obexd 0.47, 0.48 and Bluez 5 obexd 0.48 is almost the same as obexd 0.47, except that it dropped the SetFilter and SetFormat methods in favor of passing a Bluex 5-style filter parameter to PullAll. * PBAP: various enhancements for efficient caching of contacts * HTTP server: handle message resends If a client gave up waiting for the server’s response and resent its message while the server was still processing the message, syncing failed with “protocol error: already processing a message” raised by the syncevo-dbus-server because it wasn’t prepared to handle that situation. The right place to handle this is inside the syncevo-http-server, because it depends on the protocol (HTTP in this case) whether resending is valid or not. It handles that now by tracking the message that is currently in processing and matching it against each new message. If it matches, the new request replaces the obsolete one without sending the message again to syncevo-dbus-server. When syncevo-dbus-server replies to the old message, the reply is used to finish the newer request. * engine: prevent timeouts in HTTP server mode HTTP SyncML clients give up after a certain timeout (SyncEvolution after RetryDuration = 5 minutes by default, Nokia e51 after 15 minutes) when the server fails to respond. This can happen with SyncEvolution as server when it uses a slow storage with many items, for example via WebDAV. In the case of slow session startup, multithreading is now used to run the storage initializing in parallel to sending regular “keep-alive” SyncML replies to the client. By default, these replies are sent every 2 minutes. This can be configured with another extensions of the SyncMLVersion property: SyncMLVersion = REQUESTMAXTIME=5m Other modes do not use multithreading by default, but it can be enabled by setting REQUESTMAXTIME explicitly. It can be disabled by setting the time to zero. The new feature depends on a libsynthesis with multithreading enabled and glib >= 2.32.0, which is necessary to make SyncEvolution itself thread-safe. With an older glib, multithreading is disabled, but can be enabled as a stop-gap measure by setting REQUESTMAXTIME explicitly. * Various testing and stability enhancements. SyncEvolution had to be made thread-safe for the HTTP timeout prevention. * Nokia: always add TYPE=INTERNET to EMAIL ([FDO #61784](https://bugs.freedesktop.org/show_bug.cgi?id=61784)) Without the explicit TYPE=INTERNET, email addresses sent to a Nokia e51 were not shown by the phone and even got lost eventually (when syncing back). This commit ensures that the type is set for all emails sent to any Nokia phone, because there may be other phones which need it and phones which don’t, shouldn’t mind. This was spot-checked with a N97 mini, which works fine with and without the INTERNET type. This behavior can be disabled again for specific Nokia phones by adding a remote rule which sets the addInternetEmail session variable to FALSE again. Non-Nokia phones can enable the feature in a similar way, by setting the variable to TRUE. * SyncML: config option for broken peers Some peers have problems with meta data (CtCap, old Nokia phones) and the sync mode extensions required for advertising the restart capability (Oracle Beehive). The default in SyncEvolution is to advertise the capability, so manual configuration is necessary when working with a peer that fails in that mode. Because the problem occurs when SyncEvolution contacts the peers before it gets the device information from the peer, dynamic rules based on the peer identifiers cannot be used. Instead the local config must already disable these extra features in advance. The “SyncMLVersion” property gets extended for this. Instead of just “SyncMLVersion = 1.0” (as before) it now becomes possible to say “SyncMLVersion = 1.0, noctcap, norestart”. “noctcap” disables sending CtCap. “norestart” disables the sync mode extensions and thus doing multiple sync cycles in the same session (used between SyncEvolution instances in some cases to get client and server into sync in one session). Both keywords are case-insensitive. There’s no error checking for typos, so beware! The “SyncMLVersion” property was chosen because it was already in use for configuring SyncML compatibility aspects and adding a new property would have been harder. * ActiveSync: added support for specifying folder names Previously, the database field was interpreted as a Collection ID. This adds logic to allow the database to be interpreted as a folder path. The logic is: 1) If the database is an empty string, pass it through (this is the most common case as it is interpreted as “use the default folder for the source type”). 2) If the database matches a Collection ID, use the ID (this is the same as the previous behaviour). 3) If the database matches a folder path name, with an optional leading “/”, use the Collection ID for the matching folder. 4) Otherwise, force a FolderSync to get the latest folder changes from the server and repeat steps 2 and 3 5) If still no match, throw an error. * ActiveSync: support for listing databases Now –print-databases scans folders on the ActiveSync server and shows suitable folders for the ActiveSync backends instead of the previous, hard-coded help text. Invoking –print-databases can be used as a workaround for “SyncFolder error: Invalid synchronization key” errors. A better solution would be to do that automatically, but there was no time to implement that. See [FDO #61869](https://bugs.freedesktop.org/show_bug.cgi?id=61869) and “[SyncEvolution] Activesync server losing state” http://thread.gmane.org/gmane.comp.mobile.syncevolution/4295 * SyncML: workarounds for broken peers Some peers have problems with meta data (CtCap, old Nokia phones) and the sync mode extensions required for advertising the restart capability (Oracle Beehive). Because the problem occurs when SyncEvolution contacts the peers before it gets the device information from the peer, dynamic rules based on the peer identifiers cannot be used. Instead the local config must already disable these extra features in advance. The “SyncMLVersion” property gets extended for this. Instead of just “SyncMLVersion = 1.0” (as before) it now becomes possible to say “SyncMLVersion = 1.0, noctcap, norestart”. “noctcap” disables sending CtCap. “norestart” disables the sync mode extensions and thus doing multiple sync cycles in the same session (used between SyncEvolution instances in some cases to get client and server into sync in one session). Both keywords are case-insensitive. There’s no error checking for typos, so beware! The “SyncMLVersion” property was chosen because it was already in use for configuring SyncML compatibility aspects and adding a new property would have been harder. * engine: local cache sync mode This patch introduces support for true one-way syncing (“caching”): the local datastore is meant to be an exact copy of the data on the remote side. The assumption is that no modifications are ever made locally outside of syncing. This is different from one-way sync modes, which allows local changes and only temporarily disables sending them to the remote side. Another goal of the new mode is to avoid data writes as much as possible. This new mode only works on the server side of a sync, where the engine has enough control over the data flow. Setting “sync” to: - “local-cache-incremental” will do an incremental sync (if possible) or a slow sync (otherwise). This is usually the right mode to use, and thus has “local-cache” as alias. - “local-cache-slow” will always do a slow sync. Useful for debugging or after (accidentally) making changes on the local side. An incremental sync will ignore such changes because they are not meant to happen, aren’t checked for to improve performance and thus will leave client and server out-of-sync! Both modes are recorded in the sync report of the local side. The target side is the client and records the normal “two-way” or “slow” sync modes. With the current SyncEvolution contact field list, first, middle and last name are used to find matches for contacts. For events, tasks and memos, time, summary and description are used. * Minor memory leak fix when using GDBus GIO: GDBusMethodInfo Also depends on a glib fix, see [BGO #695376](https://bugzilla.gnome.org/show_bug.cgi?id=695376) * build fixes Avoid -lrt in make dependencies. Add missing pcre libs to syncevo-dbus-server. sqlite backend needs “#include <stdio.h>” (patch from Mario Kicherer). * autotools: fix temp file vulnerability during compilation ([CVE-2014-1639](http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2014-1639)) We must use the temporary file that was created for us securily, not a temp file named after that file. This caused a temp file vulnerability and the real temporary files were not deleted by the script. * workarounds for warnings from g++ 4.5 Upgrading from releases <= 1.3.99.4: ———————————— If the value of “username/databaseUser/proxyUser” contains a colon, the “user:” prefix must be added to the value, to continue treating it like a plain user name and not some reference to an unknown identity provider (like “id:”, “goa:”, “signon:”, etc.). The lookup of passwords in GNOME Keyring was updated slightly in 1.3.99.5. It may be necessary to set passwords anew if the old one is no longer found. Upgrading from release 1.2.x: —————————– The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts): syncevolution –configure \ syncFormat=text/x-vcard \ mobical addressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually: syncevolution –configure \ enableRefreshSync=TRUE \ funambol Upgrading from releases before 1.2: ———————————– Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. SyncEvolution 1.3.99.7 -> 1.4 ============================= Compared to the pre-release, 1.4 mostly just enhanced the testing. Compatibility with GNOME 3.10 and a glib-related issue that existed almost forever without causing obvious problems were fixed. syncevolution.org binaries now finally work with distros using libical.so.1 (for example, Ubuntu Saucy and Trusty). Details: * autotools: fix temp file vulnerability during compilation (CVE-2014-1639) We must use the temporary file that was created for us securily, not a temp file named after that file. This caused a temp file vulnerability and the real temporary files were not deleted by the script. * glib: fix double-free of source tags glib 2.39.0 (aka GNOME 3.10) as found in Ubuntu Trusty introduces warnings when g_source_remove() is passed an unknown tag. SyncEvolution did this in two cases: in both, the source callback returned false and thus caused the source to be removed by the caller. In that case, the explicit g_source_remove() is redundant and must be avoided. Such a call is faulty and might accidentally remove a new source with the same tag (unlikely though, given that tags seem to get assigned incrementally). The only noticable effect were additional error messages with different numbers: [ERROR] GLib: Source ID 9 was not found when attempting to remove it * libical: compatibiliy mode for libical.so.0 and libical.so.1 libical 1.0 broke the ABI, leading to libical.so.1. The only relevant change for SyncEvolution is the renumbering of ICAL_*_PROPERTY enum values. We can adapt to that change at runtime, which allows us to compile once with libical.so.0, then patch executables or use dynamic loading to run with the more recent libical.so.1 if we add 1 to the known constants. Source, Installation, Further information ========================================= Source code bundles for users are available in http://downloads.syncevolution.org/syncevolution/sources and the original source is [the git repositories](http://cgit.freedesktop.org/SyncEvolution/). i386, lpia and amd64 binaries for Debian-based distributions are available via the “stable” syncevolution.org repository. Add the following entry to your /apt/source.list:
debhttp://downloads.syncevolution.org/aptstablemain Then install “syncevolution-evolution”, “syncevolution-kde” and/or “syncevolution-activesync”. These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 10.04 LTS (Lucid), except for ActiveSync binaries which were compiled for Debian Wheezy, Ubuntu Saucy and Ubuntu Trusty. The packages mentioned above are meta-packages which pull in suitable packages matching the distro during installation. Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/). In contrast to 0.8.x archives, the 1.x .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps. More specific [HOWTOs](/wiki/howto) can be found in the Wiki.
This the final release candidate for 1.4. No further changes planned unless new problems are found. Details: * SSO: support Ubuntu Online Accounts When compiling from source on recent Ubuntu it becomes possible to use Ubuntu Online Accounts for authenticating against Google’s CalDAV and CardDAV servers. * D-Bus server: fix abort when mixing auto-sync and manual operations ([FDO #73562](https://bugs.freedesktop.org/show_bug.cgi?id=73562)) When enabling auto-sync for a config and then accessing or syncing the config manually via the command line tool, the server would abort at the time when the auto-sync was originally scheduled. * D-Bus server: accept WBXML with charset in incoming connections A user reported via email that the Nokia 515 sends ‘application/vnd.syncml+wbxml; charset=UTF-8’ as type of its messages. This tripped up the syncevo-http-server, leading to: [ERROR] syncevo-dbus-server: /org/syncevolution/Server: message type ‘application/vnd.syncml+wbxml; charset=UTf-8’ not supported for starting a sync We need to strip the ‘; charset=UTF-8’ suffix also when checking for WBXML. * packaging: compatible with EDS up to and including 3.10 The packages now contain three versions of syncecal.so: - one for EDS < 3.6 - one for EDS >= 3.6 < 3.10 - one for EDS >= 3.10 with the libecal-1.2 soname patched The third flavor became necessary because EDS 3.10 accidentally changed the soname. The API and ABI actually is the same. Package meta-data was fixed to reflect the extended range of compatible EDS libraries, so syncevolution-evolution can be installed again with recent EDS. * packaging: update syncevolution-kde dependencies kdebase-runtime became kde-runtime in Debian Wheezy. Accept both as prerequisite of syncevolution-kde to allow installation on newer distros without pulling in the transitional kdebase-runtime package. * packaging: fix rpm ([FDO #73347](https://bugs.freedesktop.org/show_bug.cgi?id=73347)) After installing the syncevolution.org rpm on OpenSUSE, SyncEvolution was not starting because its shared libraries were not found unless “ldconfig” was called manually. Now the package does that automatically. * packaging: fix description The syncevolution-bundle description of both rpm and deb packagesaccidentally used the same description as syncevolution-evolution. * test improvements, integration of cppcheck and clang’s scan-build Upgrading from release 1.2.x: ————————————- The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts): syncevolution –configure \ syncFormat=text/x-vcard \ mobical addressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually: syncevolution –configure \ enableRefreshSync=TRUE \ funambol Upgrading from releases before 1.2: ———————————————- Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Source, Installation, Further information ========================================= Source code bundles for users are available in http://downloads.syncevolution.org/syncevolution/sources and the original source is [the git repositories](http://cgit.freedesktop.org/SyncEvolution/). i386, lpia and amd64 binaries for Debian-based distributions are available via the “unstable” syncevolution.org repository. Add the following entry to your /apt/source.list:
debhttp://downloads.syncevolution.org/aptunstablemain Then install “syncevolution-evolution”, “syncevolution-kde” and/or “syncevolution-activesync”. These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 10.04 LTS (Lucid), except for “syncevolution-activesync” which depends on libraries in Debian Squeeze, for example EDS 3.4. Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/). In contrast to 0.8.x archives, the 1.x .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps. More specific [HOWTOs](/wiki/howto) can be found in the Wiki.
This update focuses on SyncEvolution in IVI again. It adds support for GENIVI Diagnostic Log and Trace (DLT) and enhances searching in the unified address book. The biggest change for normal Linux desktop users is enhanced support for recent distros. Binaries on syncevolution.org now work with EDS >= 3.6 *and* < 3.6. Distros with libical1 like Ubuntu Saucy are also supported. Automated testing was updated to cover these newer platforms more thoroughly. The binaries support Google CalDAV in combination with GNOME Online Accounts (GOA) >= 3.8 and and Google CardDAV with GOA >= 3.10. To use CardDAV with GOA 3.8, one has to [patch or recompile GOA](http://cgit.freedesktop.org/SyncEvolution/syncevolution/plain/src/backends/goa/README). Support for Google CalDAV/CardDAV with Ubuntu Online Accounts requires recompilation of SyncEvolution with [one additional patch](https://bugs.freedesktop.org/show_bug.cgi?id=72263). Details: * GNOME Online Accounts: fix D-Bus problem in syncevolution.org binaries Support was included in syncevolution.org binaries, but was not tested and did not actually work due to some issue accessing the D-Bus session. * libsynthesis: partial fix batching of items The batching of contact writes introduced with SyncEvolution 1.3.99.4 caused problems with non-SyncEvolution SyncML peers when syncing contacts stored in EDS >= 3.6. EDS < 3.6 was not affected. That part is fixed. However, even in SyncEvolution<->SyncEvolution syncs another crash was found. This will require more investigation. Clearly the feature is not ready yet for general sync, so for now it is disabled by default and only enabled in the simpler PBAP sync. * libsynthesis: avoid redundant (and sometimes slow) getaddrbyname() ([FDO #70771](https://bugs.freedesktop.org/show_bug.cgi?id=70771)) The network lookup of the hostname can be slow (10 second delay when not connected) and shouldn’t be necessary anyway, so disable it. * PIM: accent-insensitive and transliterated search ([FDO #56524](https://bugs.freedesktop.org/show_bug.cgi?id=56524)) Accent-insensitive search ignores accents, using the same code as in EDS. Transliterated search ignores foreign scripts by transliterating search term and contact properties to Latin first. That one is using ICU directly in the same way as EDS, but doesn’t use the EDS ETransliterator class to avoid extra string copying. This commit changes the default behavior such that searching is by default most permissive (case- and accent-insensitive, does transliteration). Flags exist to restore more restrictive matching. * PIM: relax phone number matching Previously, the current default country was used to turn phone numbers without an explicit country code into full E164 numbers, which then had to match the search term when doing a caller ID lookup. This was inconsistent with EDS, where a weaker EQUALS_NATIONAL_PHONE_NUMBER was done. The difference is that a comparison between a number with country code matches one without if the national number of the same, regardless of the current default country. This is better because it reduces the influence of the hard to guess default country on matching. Another advantage of this change is the lower memory consumption and faster comparison, because strings are now stored in 4 + 8 byte numbers instead of strings of varying length. * PIM: fix incorrect write into pim-manager.ini ([FDO #70772](https://bugs.freedesktop.org/show_bug.cgi?id=70772)) Removing a peer accidentally wrote the updated list of active address books into the “sort” property of pim-manager.ini, which then prevented starting the PIM Manager. * PIM: ignore broken sort order in config ([FDO #70772](https://bugs.freedesktop.org/show_bug.cgi?id=70772)) Failure to set the sort order from pim-manager.ini should not prevent the startup of the PIM Manager because the client cannot really diagnose and fix the problem. It is better to try again with the default sort order. * PIM: adapt to locale changes at runtime ([FDO #66618](https://bugs.freedesktop.org/show_bug.cgi?id=66618)) Listen to signals from localed D-Bus system service and update all internal state which depends on the current locale. This state includes: - pre-computed data in all loaded contacts - filtering (for example, case sensitivity is locale dependent) - the sort order This feature can be controlled by setting the SYNCEVOLUTION_LOCALED env variable: - “session” - use a localed instance on the D-Bus session bus instead of the system instance. This was originally meant for testing, but might also be useful for per-user setting changes. - “none” - disables the feature * PIM: fix sync.py + multiple peers Due to overwriting a variable, configuring multiple different peers did not work. * D-Bus server: support DLT ([FDO #66769](https://bugs.freedesktop.org/show_bug.cgi?id=66769)) Diagnostic Log and Trace (DLT) manages a sequence of log messages, with remote controllable level of detail. SyncEvolution optionally (can be chosen at compile time and again at runtime) uses DLT instead of its own syncevolution-log.html files. See README-DLT.rst for more information. To use the feature, configure SyncEvolution with “–enable-dbus-server=–dlt –no-syslog” * EDS: enhanced compatibility mode SyncEvolution compiled for EDS < 3.6 can now also load EDS backends compiled for EDS >= 3.6. The packaging for syncevolution.org uses that to bundle EDS backends compiled on different distros in the same package. * EDS: SYNCEVOLUTION_EBOOK_QUERY env variable Setting the SYNCEVOLUTION_EBOOK_QUERY env variable to a valid EBook query string limits the results to contacts matching that query. Useful only in combination with –print-items or –export. Only implemented for EDS >= 3.6. * EDS: fix compile problem with boost and EDS > 3.36 This fixes the following problem, seen with Boost 1.53.0 on altlinux when compiling for EDS >= 3.6: /usr/include/boost/smart_ptr/shared_ptr.hpp: In instantiation of ‘typename boost::detail::sp_array_access::type boost::shared_ptr::operator const [with T = char*; typename boost::detail::sp_array_access::type = void; std::ptrdiff_t = long int]’: src/backends/evolution/EvolutionSyncSource.cpp:163:38: required from here /usr/include/boost/smart_ptr/shared_ptr.hpp:663:22: error: return-statement with a value, in function returning ‘void’ [-fpermissive] make[2]: *** [src/backends/evolution/src_backends_evolution_syncecal_la-EvolutionSyncSource.lo] * PBAP: add support for obexd 0.48 obexd 0.48 is almost the same as obexd 0.47, except that it dropped the SetFilter and SetFormat methods in favor of passing a Bluex 5-style filter parameter to PullAll. SyncEvolution now supports 4, in words, four different obexd APIs. Sigh. This feature was originally announced for SyncEvolution 1.3.99.5, but not actually included in the code yet. Upgrading from release 1.2.x: ————————————- The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts): syncevolution –configure \ syncFormat=text/x-vcard \ mobical addressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually: syncevolution –configure \ enableRefreshSync=TRUE \ funambol Upgrading from releases before 1.2: ———————————————- Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Source, Installation, Further information ========================================= Source code bundles for users are available in http://downloads.syncevolution.org/syncevolution/sources and the original source is [the git repositories](http://cgit.freedesktop.org/SyncEvolution/). i386, lpia and amd64 binaries for Debian-based distributions are available via the “unstable” syncevolution.org repository. Add the following entry to your /apt/source.list:
debhttp://downloads.syncevolution.org/aptunstablemain Then install “syncevolution-evolution”, “syncevolution-kde” and/or “syncevolution-activesync”. These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 10.04 LTS (Lucid), except for “syncevolution-activesync” which depends on libraries in Debian Squeeze, for example EDS 3.4. Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/). In contrast to 0.8.x archives, the 1.x .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps. More specific [HOWTOs](/wiki/howto) can be found in the Wiki.
I just presented the work done on PIM in Tizen IVI 3.0 at the Linux Foundation’s Automotive Linux Summit. The work is based on a considerably enhanced Evolution Data Server, libphonenumber, folks, and of course SyncEvolution. If you want to know more about this, the slides are attached.
SyncEvolution now supports Google CalDAV/CardDAV with OAuth2 authentication. These are the open protocol that Google currently supports and thus the recommended way of syncing with Google, replacing ActiveSync and SyncML (both no longer available to all Google customers). Support for Google CardDAV is new. Because of a vCard encoding issue on the server side, spaces in long notes may get removed. Like Evolution, SyncEvolution does not yet support some of the advanced features of the server, in particular custom labels for phone numbers, emails and addresses. Likewise, some client properties are not supported by the server: CALURI, CATEGORIES, FBURL, GEO and ROLE are not supported. Of ORG, only the first two components are supported. Currently, properties not supported by one side get lost in a full roundtrip sync. SyncEvolution depends on external components for OAuth2. It can be compiled to use [gSSO](http://cgit.freedesktop.org/SyncEvolution/syncevolution/plain/src/backends/signon/README) or [GNOME Online Accounts](http://cgit.freedesktop.org/SyncEvolution/syncevolution/plain/src/backends/goa/README). GNOME Online Accounts >= 3.10 works out of the box for CalDAV and CardDAV, 3.8 only for CardDAV (but the GNOME Online Accounts binary can be patched to also support CalDAV), anything older than 3.8 does not work. Support for Ubuntu Online Accounts should not be hard to add, but is [not available yet](http://thread.gmane.org/gmane.comp.mobile.syncevolution/4353/focus=4490). Evolution >= 3.6 is not supported by the binaries on syncevolution.org. On systems with a more recent Evolution, SyncEvolution must be compiled from source. Details: * GTK UI: fixed two crashes - running a sync with no service selected and a 64 bit pointer problem recently discovered by Tino Keitel when compiling the Debian package with -fPIE. * password handling: fix usage of GNOME Keyring and KWallet ([FDO #66110](https://bugs.freedesktop.org/show_bug.cgi?id=66110)) When clients like the GTK sync-ui stored a password, it was always stored as plain text in the config.ini file by the syncevo-dbus-server. The necessary code for redirecting the password storage in a keyring (GNOME or KWallet) simply wasn’t called in that case. The command line tool, even when using the D-Bus server to run the operation, had the necessary code active and thus was not affected. Now all SyncEvolution components use the same default: use safe password storage if either GNOME Keyring or KWallet were enabled during compilation, don’t use it if not. Fixing this revealed other problems, like not being able to store certain passwords that lacked the necessary lookup criteria (like syncURL and/or username). To address this, the lookup criteria where extended and a new check was added to avoid accidentally removing other passwords. As a result, it may be possible that SyncEvolution no longer finds passwords that were stored with older versions of SyncEvolution. In such a case the passwords must be set again. * GNOME: clean up keyring access and require libgnome-keyring >= 2.20 The updated error messages now always include information about the password and libgnome-keyring error texts. A workaround is used for the “Error communicating with gnome-keyring-daemon” problem that started to appear fairly frequently in the automated testing once the keyring was actually used. The problem shows up with some additional debug messages: Gkr: received an invalid, unencryptable, or non-utf8 secret Gkr: call to daemon returned an invalid response: (null).(null)() It seems that sometimes setting up a session with GNOME keyring fails such that all further communication leads to decoding problem. There is an internal method to reset the session, but it cannot be called directly. As a workaround, fake the death of the GNOME keyring daemon and thus trigger a reconnect when retrying the GNOME keyring access. This is done by sending a D-Bus message, which will also affect other clients of GNOME keyring, but hopefully without user-visible effects. * config: enhanced password handling It is possible to configure a plain username/password combination once in SyncEvolution and then use references to it in other configurations, instead of having to set (and update) the credentials in different places. This is useful in particular with WebDAV, where credentials had to be repeated several times (target config, in each database when used as part of SyncML) or when using a service which requires several configs (Google via SyncML and CalDAV). To use this, create a sync config for a normal peer or a dedicated config just for the credentials, with “username/password/syncURL” set. The “syncURL” must be set to something identifying the peer if GNOME Keyring is used for the password storage. Then set “username”, “databaseUser” and “proxyUser” properties to “id:” and all read and write access to those properties will be redirected by SyncEvolution into that other configuration. This even works in the GTK UI. For user names which contain colons, the new “user:” format must be used. Strings without colons are assumed to be normal user names, so most old configurations should continue to work. * signon: new backend using libgsignond-glib + libaccounts-glib The code works with gSSO (https://01.org/gsso). With some tweaks to the configure check and some ifdefs it probably could be made to work with Ubuntu Online Accounts. The code depends on an account accessible via libaccounts-glib which has a provider and and (optionally) services enabled for that provider. It is not necessary that the account already has a signon identity ID, the backend will create that for the provider (and thus shared between all services) if necessary. Therefore it is possible to use the ag-tool to create and enable the account and services. Provider and service templates are in the next commit. * WebDAV: support OAuth2 If given an authentication configuration which can handle OAuth2, then OAuth2 is used instead of plain username/password authentication. * WebDAV: support Google CardDAV, break Yahoo Google CardDAV has one peculiarity: it renames new contacts during PUT without returning the new path to the client. See also http://lists.calconnect.org/pipermail/caldeveloper-l/2013-July/000524.html SyncEvolution already had a workaround for that (PROPGET on old path, extract new path from response) which happened to work. This workaround was originally added for Yahoo, which sometimes merges contacts into existing ones. In contrast to Yahoo, Google really seems to create new items. Without some server specific hacks, the client cannot tell what happened. Because Google is currently supported and Yahoo is not, let’s change the hard-coded behavior to “renamed items are new”. * WebDAV: started testing with owndrive.com = OwnCloud * GOA: get OAuth2 tokens out of GNOME Online Accounts “username = goa:…” selects an account in GOA and retrieves the OAuth2 token from that. The implementation uses the GOA D-Bus API directly, because our C++ D-Bus bindings are easier to use and this avoids an additional library dependency. * PIM: fix UID usage in sync.py example Using the underscore in the UID has been wrong all along, it only happened to work because UID sanity checking was missing. After adding it, the example broke. Now simply remove the colon. It makes the UID less readable, but it doesn’t have to be, and ensures that file names and database names contain the UID as-is. * PIM: if busy, don’t shut down While there are sessions pending or active, the server should not shut down. It did that while executing a long-running PIM Manager SyncPeer() operations, by default after 10 minutes. This was not a problem elsewhere because other operations are associated with a client, whose presence also prevents shutdowns. Perhaps PIM Manager should also track the caller and treat it like a client. * PBAP: do not end Bluez5 transfer prematurely A transfer was marked as finished prematurely when encountering the “active” Status value, which can happen for longer transfers. * PBAP: add support for obexd 0.48 obexd 0.48 is almost the same as obexd 0.47, except that it dropped the SetFilter and SetFormat methods in favor of passing a Bluex 5-style filter parameter to PullAll. SyncEvolution now supports 4, in words, four different obexd APIs. Sigh. * updated tests Upgrading from releases <= 1.3.99.4: ———————————————– If the value of “username/databaseUser/proxyUser” contains a colon, the “user:” prefix must be added to the value, to continue treating it like a plain user name and not some reference to an unknown identity provider (like “id:”, “goa:”, “signon:”, etc.). The lookup of passwords in GNOME Keyring was updated slightly in 1.3.99.5. It may be necessary to set passwords anew if the old one is no longer found. Upgrading from release 1.2.x: ————————————- The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts): syncevolution –configure \ syncFormat=text/x-vcard \ mobical addressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually: syncevolution –configure \ enableRefreshSync=TRUE \ funambol Upgrading from releases before 1.2: ———————————————- Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Source, Installation, Further information ========================================= Source code bundles for users are available in http://downloads.syncevolution.org/syncevolution/sources and the original source is [the git repositories](http://cgit.freedesktop.org/SyncEvolution/). i386, lpia and amd64 binaries for Debian-based distributions are available via the “unstable” syncevolution.org repository. Add the following entry to your /apt/source.list:
debhttp://downloads.syncevolution.org/aptunstablemain Then install “syncevolution-evolution”, “syncevolution-kde” and/or “syncevolution-activesync”. These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 10.04 LTS (Lucid), except for “syncevolution-activesync” which depends on libraries in Debian Squeeze, for example EDS 3.4. Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/). In contrast to 0.8.x archives, the 1.x .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps. More specific [HOWTOs](/wiki/howto) can be found in the Wiki.
The focus of this development snapshot is enhanced performance of syncing. With EDS, contacts get added, updated or loaded with batch operations, which led to 4x runtime improvements when importing PBAP address book for the first time. Removing unnecessary work from any following PBAP sync resulted in a 6x improvement. These improvements also benefit non-PBAP syncing and could in theory work with any SyncML peer. In practice, batching of items is currently limited to SyncEvolution as peer. The PBAP backend itself was rewritten such that data gets transferred from a phone in parallel to processing the already transferred data. The effect is that on a sufficiently fast system, a sync takes about the same time as downloading all contacts. To get the text-only part of the contacts even faster, PBAP syncing can be done such that it first syncs the text-only parts (without removing existing photos), then in a second round adds or modifies photos. The PIM Manager uses this incremental mode by default, in the command line it can be chose with the SYNCEVOLUTION_PBAP_SYNC env variable. The HTTP server became better at handling message resends when the server is slow with processing a message. The server is able to keep a sync session alive while loading the initial data set by sending acknowledgement replies before the client times out. Guido Günther provided some patches addressing problems when compiling SyncEvolution for Maemo. Details: * sync: less verbose output, shorter runtime For each incoming change, one INFO line with “received x[/out of y]” was printed, immediately followed by another line with total counts “added x, updated y, removed z”. For each outgoing change, a “sent x[/out of y]” was printed. In addition, these changes were forwarded to the D-Bus server where a “percent complete” was calculated and broadcasted to clients. All of that caused a very high overhead for every single change, even if the actual logging was off. The syncevo-dbus-server was constantly consuming CPU time during a sync when it should have been mostly idle. To avoid this overhead, the updated received/sent numbers that come from the Synthesis engine are now cached and only processed when done with a SyncML message or some other event happens (whatever happens first). To keep the implementation simple, the “added x, updated y, removed z” information is ignored completely and no longer appears in the output. * HTTP server: handle message resends If a client gave up waiting for the server’s response and resent its message while the server was still processing the message, syncing failed with “protocol error: already processing a message” raised by the syncevo-dbus-server because it wasn’t prepared to handle that situation. The right place to handle this is inside the syncevo-http-server, because it depends on the protocol (HTTP in this case) whether resending is valid or not. It handles that now by tracking the message that is currently in processing and matching it against each new message. If it matches, the new request replaces the obsolete one without sending the message again to syncevo-dbus-server. When syncevo-dbus-server replies to the old message, the reply is used to finish the newer request. * PBAP: incremental sync ((FDO #59551)[https://bugs.freedesktop.org/show_bug.cgi?id=59551]) Depending on the SYNCEVOLUTION_PBAP_SYNC env variable, syncing reads all properties as configured (“all”), excludes photos (“text”) or first text, then all (“incremental”). When excluding photos, only known properties get requested. This avoids issues with phones which reject the request when enabling properties via the bit flags. This also helps with “databaseFormat=^PHOTO”. * PIM: use incremental sync for PBAP by default ((FDO #59551)[https://bugs.freedesktop.org/show_bug.cgi?id=59551]) When doing a PBAP sync, PIM manager asks the D-Bus sync helper to set its SYNCEVOLUTION_PBAP_SYNC to “incremental”. If the env variable is already set, it does not get overwritten, which allows overriding this default. * PIM: set debug level in peer configs via env variable Typically the peer configs get created from scratch, in particular when testing with testpim.py. In that case the log level cannot be set in advance and doing it via the D-Bus API is also not supported. Therefore, for debugging, use SYNCEVOLUTION_LOGLEVEL= to create peers with a specific log level. * PIM: include pim-manager-api.txt in source distro ((FDO #62516)[https://bugs.freedesktop.org/show_bug.cgi?id=62516]) The text file must be listed explicitly to be included by “make dist”. * PIM: “full name” -> “fullname” fix in documentation ((FDO #62515)[https://bugs.freedesktop.org/show_bug.cgi?id=62515]) Make the documentation match the code. A single word without space makes more sense, so let’s go with what the code already used. * PIM: enhanced searching (search part of (FDO #64177)[https://bugs.freedesktop.org/show_bug.cgi?id=64177]) Search terms now also include ‘is/contains/begins-with/ends-with’ and they can be combined with ‘and’ and ‘or’, also recursively. * PIM: Pinyin sorting for zh languages (part of (FDO #64173)[https://bugs.freedesktop.org/show_bug.cgi?id=64173]) Full interleaving of Pinyin transliterations of Chinese names with Western names can be done by doing an explicit Pinyin transliteration as part of computing the sort keys. This is done using ICU’s Transliteration(“Han-Latin”), which we have to call directly because boost::locale does not expose that API. We hard-code this behavior for all “zh” languages (as identified by boost::locale), because by default, ICU would sort Pinyin separately from Western names when using the “pinyin” collation. * PIM: new return value for SyncPeer(), new SyncProgress signal ((FDO #63417)[https://bugs.freedesktop.org/show_bug.cgi?id=63417]) The SyncPeer() result is derived from the sync statistics. To have them available, the “sync done” signal must include the SyncReport. Start and end of a sync could already be detected; “modified” signals while a sync runs depends on a new signal inside the SyncContext when switching from one cycle to the next and at the end of the last one. * PIM: allow removal of data together with database removal (part of (FDO #64835)[https://bugs.freedesktop.org/show_bug.cgi?id=64835]) There is a difference in EDS between removing the database definition from the ESourceRegistry (which makes the data unaccessible via EDS) and removing the actual database. EDS itself only removes the definition and leaves the data around to be garbage-collected eventually. This is not what we want for the PIM Manager API; the API makes a stronger guarantee that data is really gone. Fixed by introducing a new mode flag for the deleteDatabase() method and deleting the directory of the source directly in the EDS backend, if requested by the caller. The syncevolution command line tool will use the default mode and thus keep the data around, while the PIM Manager forces the removal of data. * EDS: create new databases by cloning the builtin ones ((FDO #64176)[https://bugs.freedesktop.org/show_bug.cgi?id=64176]) Instead of hard-coding a specific “Backend Summary Setup” in SyncEvolution, copy the config of the system database. That way special flags (like the desired “Backend Summary Setup” for local address books) can be set on a system-wide basis and without having to modify or configure SyncEvolution. Because EDS has no APIs to clone an ESource or turn a .source file into a new ESource, SyncEvolution has to resort to manipulating and creating the keyfile directly. * EDS contacts: update PHOTO+GEO during slow sync, avoid rewriting PHOTO file If PHOTO and/or GEO were the only modified properties during a slow sync, the updated item was not written into local storage because they were marked as compare=”never” = “not relevant”. For PHOTO this was intentional in the sample config, with the rationale that local storages often don’t store the data exactly as requested. When that happens, comparing the data would lead to unnecessary writes. But EDS and probably all other local SyncEvolution storages (KDE, file) store the photo exactly as requested, so not considering changes had the undesirable effect of not always writing new photo data. For GEO, ignoring it was accidental. * EDS contacts: avoid unnecessary DB writes during slow sync Traditionally, contacts were modified shortly before writing into EDS to match with Evolution expectations (must have N, only one CELL TEL, VOICE flag must be set). During a slow sync, the engine compare the modified contacts with the unmodified, incoming one. This led to mismatches and/or merge operations which end up not changing anything in the DB because the only difference would be removed again before writing. * EDS contacts: read-ahead cache Performance is improved by requesting multiple contacts at once and overlapping reading with processing. On a fast system (SSD, CPU fast enough to not be the limiting factor), testpim.py’s testSync takes 8 seconds for a “match” sync where 1000 contacts get loaded and compared against the same set of contacts. Read-ahead with only 1 contact per query speeds that up to 6.7s due to overlapping IO and processing. Read-ahead with the default 50 contacts per query takes 5.5s. It does not get much faster with larger queries. * command line: execute –export and –print-items while the source is still reading Instead of reading all item IDs, then iterating over them, process each new ID as soon as it is available. With sources that support incremental reading (only the PBAP source at the moment) that provides output sooner and is a bit more memory efficient. * WebDAV: avoid segfault during collection lookup Avoid referencing pathProps->second when the set of paths that PROPFINDs returns is empty. Apparently this can happen in combination with Calypso. * engine: prevent timeouts in HTTP server mode HTTP SyncML clients give up after a certain timeout (SyncEvolution after RetryDuration = 5 minutes by default, Nokia e51 after 15 minutes) when the server fails to respond. This can happen with SyncEvolution as server when it uses a slow storage with many items, for example via WebDAV. In the case of slow session startup, multithreading is now used to run the storage initializing in parallel to sending regular “keep-alive” SyncML replies to the client. By default, these replies are sent every 2 minutes. This can be configured with another extensions of the SyncMLVersion property: SyncMLVersion = REQUESTMAXTIME=5m Other modes do not use multithreading by default, but it can be enabled by setting REQUESTMAXTIME explicitly. It can be disabled by setting the time to zero. The new feature depends on a libsynthesis with multithreading enabled and glib >= 2.32.0, which is necessary to make SyncEvolution itself thread-safe. With an older glib, multithreading is disabled, but can be enabled as a stop-gap measure by setting REQUESTMAXTIME explicitly. * Various testing and stability enhancements. SyncEvolution had to be made thread-safe for the HTTP timeout prevention. Source, Installation, Further information ========================================= Source code bundles for users are available in http://downloads.syncevolution.org/syncevolution/sources and the original source is [the git repositories](http://cgit.freedesktop.org/SyncEvolution/). i386, lpia and amd64 binaries for Debian-based distributions are available via the “unstable” syncevolution.org repository. Add the following entry to your /apt/source.list:
debhttp://downloads.syncevolution.org/aptunstablemain Then install “syncevolution-evolution”, “syncevolution-kde” and/or “syncevolution-activesync”. These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 10.04 LTS (Lucid), except for “syncevolution-activesync” which depends on libraries in Debian Squeeze, for example EDS 3.4. Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/). In contrast to 0.8.x archives, the 1.x .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps. More specific [HOWTOs](/wiki/howto) can be found in the Wiki.
So far the development cycle for SyncEvolution 1.4 mostly focused on implementing the [“PIM Manager”](http://comments.gmane.org/gmane.comp.mobile.syncevolution/4009) D-Bus API for IVI use cases [1]. The 1.3.99.3 pre-release starts to include more features and bug fixes again for syncing. For example, several ActiveSync improvements from Graham Cobb were included. The remaining goal for 1.4, besides more testing of course, is to work out how to support Google CalDAV and CardDAV. I am in discussion with Google to get SyncEvolution [whitelisted](http://googleblog.blogspot.de/2013/03/a-second-spring-of-cleaning.html) for use with these APIs - fingers crossed… Upgrading from release 1.2.x —————————- The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts): syncevolution –configure \ syncFormat=text/x-vcard \ mobical addressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually: syncevolution –configure \ enableRefreshSync=TRUE \ funambol Upgrading from releases before 1.2 ———————————- Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Changes 1.3.2 -> 1.3.99.3 ========================= * PIM Manager: add ReplaceSearch, always allow it The new ReplaceSearch is more flexible than RefineSearch. It can handle both tightening the search and relaxing it. The downside of it is the more expensive implementation (must check all contacts again, then find minimal set of change signals to update view). Previously, a search which had no filter set at all at the begining could not be refined. This limitation of the implementation gets removed by always using a FilteredView, even if the initial filter is empty. * PIM Manager: introduce CreateConfig() That SetPeer() allows modifying and creating a config leads to race conditions when multiple clients want to create a config. The new CreateConfig() avoids that by atomically checking that a config does not exist yet and creating it. SetPeer() is still available for backwards compatibility. It continues to be used for modifying an existing config in TestContacts.testSync to check the effect of the logging settings. * PIM Manager: fix double entries in filtered search with limit Stressing the FilteredView by using it in tests originally written for the FullView showed that the filling up a view may have used data while it was inconsistent internally, leading to contacts being present multiple times. * PIM Manager and sync: support location = GEO property ([FDO #60373](https://bugs.freedesktop.org/show_bug.cgi?id=60373)) Exposed as “location” -> (lat, long) in the D-Bus bindings. Reading, writing and updating are supported. * PIM Manager: support groups = CATEGORIES ([FDO #60380](https://bugs.freedesktop.org/show_bug.cgi?id=60380)) Allow reading and writing of groups (folks terminology), aka CATEGORIES in vCard. * PIM Manager: intelligent phone search in EDS (part of [FDO #59571](https://bugs.freedesktop.org/show_bug.cgi?id=59571)) If phone number search is enabled in EDS, then the direct search in EDS now uses the more accurate E_BOOK_QUERY_EQUALS_NATIONAL_PHONE_NUMBER comparison, with the E164 formatted caller ID as value to compare against. This gives semantically correct results. The previous solution (now the fallback) had to use substring searches, which did not match if the contact’s phone number was not formatted according to E164 and which may have matched the wrong contacts if the trailing numbers are the same. * PIM Manager : use pre-computed normalized phone numbers from EDS (part of [FDO #59571](https://bugs.freedesktop.org/show_bug.cgi?id=59571)) When available, the pre-computed E164 number from EDS will be used instead of doing one libphonebook parser run for each telephone number while reading. Benchmarking showed that this parsing was the number one hotspot, so this is a considerable improvement. * PIM Manager: fix error messages Ensure and check that no unnecessary ERROR messages are printed. libfolks was used slightly incorrectly, leading to several harmless error messages (glib asserts). libphonenumber printed its error messages to stdout. * PIM Manager: fix memory leaks during writing of contacts Constructing the GValues created additional references instead of taking over ownership as intended. * D-Bus server: fix read-after-free bug when using syslog openlog() expects the string to remain valid. Must ensure that in LoggerSyslog by making a copy. Found with valgrind. * PIM Manager: make implementation of some of the D-Bus methods thread-safe The goal is to make it easier to extend syncevo-dbus-server with other IPC mechanisms, which then can call the native C++ code directly. That code was not prepared to handle calls in threads other than the main one. Now this is checked when entering the methods and work is shifted to the main thread if necessary. In the meantime the calling thread waits for completion. * PIM Manager: check responsiveness (part of [FDO #60851](https://bugs.freedesktop.org/show_bug.cgi?id=60851)) Enhanced the testActive test so that it can detect when the D-Bus server stops responding for too long. One major reason for that was event processing in folks, which got improved as part of https://bugzilla.gnome.org/show_bug.cgi?id=694385 * PIM Manager: adapt to gee 0.8 Changed the code to compile with gee 0.8, as used by folks 0.9.x. Older versions of folks are no longer supported. * PBAP: support Bluez 5 The new Bluez 5 API is the third supported API for doing PBAP transfers. It gets checked first, then the PBAB backend falls back to new-style obexd (file based, similar to Bluez 5, but not quite the same) and finally old-style obexd (data transfer via D-Bus). In contrast to previous APIs, Bluez 5 does not report the reason for a failed PBAP transfer. SyncEvolution then throws a generic “transfer failed” error with “reason unknown” as message. * command line: recover from slow sync with new sync modes The error message for an unexpected slow sync still mentioned the old and obsolete “refresh-from-client/server” sync modes. Better mention “refresh-from-local/remote”. * CalDAV: more workarounds for Google CalDAV + unique IDs Google became even more strict about checking REV. Tests which reused a UID after deleting the original item started to fail sometime since middle of December 2012. * CalDAV: work around Google server regression (undeclared namespace prefix in XML) Google CalDAV for a while (December 2012 till January 2013) sent invalid XML back when asked to include CardDAV properties in a PROPFIND. This got rejected in the XML parser, which prevents syncing calendar data: Neon error code 1: XML parse error at line 55: undeclared namespace prefix In the meantime Google fixed the issue in response to a bug report via email. But the workaround, only asking for the properties which are really needed, still makes sense and thus is kept. * WebDAV: don’t send Basic Auth via http proactively ([FDO #57248](https://bugs.freedesktop.org/show_bug.cgi?id=57248)) Sending basic authentication headers via http is insecure. Only do it proactively when the connection is encrypted and thus protects the information or when the server explicitly asks for it. * Nokia: always add TYPE=INTERNET to EMAIL ([FDO #61784](https://bugs.freedesktop.org/show_bug.cgi?id=61784)) Without the explicit TYPE=INTERNET, email addresses sent to a Nokia e51 were not shown by the phone and even got lost eventually (when syncing back). This commit ensures that the type is set for all emails sent to any Nokia phone, because there may be other phones which need it and phones which don’t, shouldn’t mind. This was spot-checked with a N97 mini, which works fine with and without the INTERNET type. This behavior can be disabled again for specific Nokia phones by adding a remote rule which sets the addInternetEmail session variable to FALSE again. Non-Nokia phones can enable the feature in a similar way, by setting the variable to TRUE. * SyncML: config option for broken peers Some peers have problems with meta data (CtCap, old Nokia phones) and the sync mode extensions required for advertising the restart capability (Oracle Beehive). The default in SyncEvolution is to advertise the capability, so manual configuration is necessary when working with a peer that fails in that mode. Because the problem occurs when SyncEvolution contacts the peers before it gets the device information from the peer, dynamic rules based on the peer identifiers cannot be used. Instead the local config must already disable these extra features in advance. The “SyncMLVersion” property gets extended for this. Instead of just “SyncMLVersion = 1.0” (as before) it now becomes possible to say “SyncMLVersion = 1.0, noctcap, norestart”. “noctcap” disables sending CtCap. “norestart” disables the sync mode extensions and thus doing multiple sync cycles in the same session (used between SyncEvolution instances in some cases to get client and server into sync in one session). Both keywords are case-insensitive. There’s no error checking for typos, so beware! The “SyncMLVersion” property was chosen because it was already in use for configuring SyncML compatibility aspects and adding a new property would have been harder. * ActiveSync: added support for specifying folder names Previously, the database field was interpreted as a Collection ID. This adds logic to allow the database to be interpreted as a folder path. The logic is: 1) If the database is an empty string, pass it through (this is the most common case as it is interpreted as “use the default folder for the source type”). 2) If the database matches a Collection ID, use the ID (this is the same as the previous behaviour). 3) If the database matches a folder path name, with an optional leading “/”, use the Collection ID for the matching folder. 4) Otherwise, force a FolderSync to get the latest folder changes from the server and repeat steps 2 and 3 5) If still no match, throw an error. * ActiveSync: support for listing databases Now –print-databases scans folders on the ActiveSync server and shows suitable folders for the ActiveSync backends instead of the previous, hard-coded help text. Invoking –print-databases can be used as a workaround for “SyncFolder error: Invalid synchronization key” errors. A better solution would be to do that automatically, but there was no time to implement that. See [FDO #61869](https://bugs.freedesktop.org/show_bug.cgi?id=61869) and “[SyncEvolution] Activesync server losing state” http://thread.gmane.org/gmane.comp.mobile.syncevolution/4295 * command line: show backend error when listing databases fails The command line swallowed errors thrown by the backend while listing databases. Instead it just showed “: backend failed”. The goal was to not distract users who accidentally access a non-functional backend. But the result is that operations like –configure or –print-databases could fail without giving the user any hint about the root cause of the issue. Now the error explanation in all its gory details is included. For example, not having activesyncd running leads to: INFO] eas_contact: backend failed: fetching folder list: GDBus.Error:org.freedesktop.DBus.Error.ServiceUnknown: The name org.meego.activesyncd was not provided by any .service files And running activesyncd without the necessary gconf keys shows up as: [INFO] eas_contact: backend failed: fetching folder list: GDBus.Error:org.meego.activesyncd.Error.AccountNotFound: Failed to find account [syncevolution@lists.intel.com] * Minor memory leak fix when using GDBus GIO: GDBusMethodInfo Also depends on a glib fix, see https://bugzilla.gnome.org/show_bug.cgi?id=695376 * build fixes Avoid -lrt in make dependencies. Add missing pcre libs to syncevo-dbus-server. sqlite backend needs “#include <stdio.h>” (patch from Mario Kicherer). 1.3.99.1 => 1.3.99.2, 13.12.2012 ================================ * PIM Manager searches for a caller ID (‘phone’ search) in EDS directly while folks still starts up. No unification is done of these results. Intermediate results are replaced by the final ones from folks once those are ready. * PIM Manager: allow configuration of session directories (part of [FDO #55921](https://bugs.freedesktop.org/show_bug.cgi?id=55921)) Useful for moving the session directories to a temporary file system. They are essentially just useful for debugging when used as part of PIM Manager. - “logdir” - a directory in which directories are created with debug information about sync session - “maxsessions” - number of sessions that are allowed to exist after a sync (>= 0): 0 is special and means unlimited, 1 for just the latest, etc.; old sessions are pruned heuristically (for example, keep sessions where something changed instead of some where nothing changed), so there is no hard guarantee that the last n sessions are present. * PIM Manager: write less data to disk (part of [FDO #55921](https://bugs.freedesktop.org/show_bug.cgi?id=55921)) Avoid writing config file changes to disk by enabling a new “ephemeral” mode for syncing via the PIM Manager. In this mode, config file changes are not flushed resp. discarded directly. This prevents writing to .ini files in ~/.config. The “synthesis” binfile client files are still written, but they get redirected into the session directory, which can (and should) be set to a temp file system and get deleted again quickly. Data dumps are turned off now in the configs created by the PIM Manager. * syncevo-dbus-server: use syslog instead of standard output by default * syncevo-dbus-server: command line options for controlling output and startup -d, –duration=seconds/’unlimited’ Shut down automatically when idle for this duration (default 300 seconds) -v, –verbosity=level Choose amount of output, 0 = no output, 1 = errors, 2 = info, 3 = debug; default is 1. -o, –stdout Enable printing to stdout (result of operations) and stderr (errors/info/debug). -s, –no-syslog Disable printing to syslog. -p, –start-pim Activate the PIM Manager (= unified address book) immediately. * PIM Manager: store set of active address books persistently ([FDO #56334](https://bugs.freedesktop.org/show_bug.cgi?id=56334)) Together with storing the sort order persistently, this allows restarting the daemon and have it create the same unified address book again. * PIM Manager: remove colon from valid peer UID character set ([FDO #56436](https://bugs.freedesktop.org/show_bug.cgi?id=56436)) Using the UID as part of file names gets more problematic when allowing colons. Remove that character from the API and enforce the format in the source code. * PIM Manager API: introduce contact ID and use it for reading This makes it easier for a client to fully polulate its view with contact data. Previously it could happen that due to concurrent changes in the server, a client was returned data for the same contact multiple times. A client had to detect that and re-issue read requests. * PIM Manager API: optional ViewAgent.Quiescent() ([FDO #56428](https://bugs.freedesktop.org/show_bug.cgi?id=56428)) The callback is guaranteed to be invoked once when a search has finished sending its initial results, and not sooner. This makes it possible to check whether the current data contains some contact or not. * PIM Manager: limit number of search results ([FDO #56142](https://bugs.freedesktop.org/show_bug.cgi?id=56142)) A ‘limit’ search term with a number as parameter (formatted as string) can be added to a ‘phone’ or ‘any-contains’ search term to truncate the search results after a certain number of contacts. Example: Search([[‘any-contains’, ‘Joe’], [‘limit’, ‘10’]]) => return the first 10 Joes. As with any other search, the resulting view will be updated if contact data changes. The limit must not be changed in a RefineSearch(). A ‘limit’ term may (but doesn’t have to) be given. If it is given, its value must match the value set when creating the search. This limitation simplifies the implementation and its testing. The limitation could be removed if there is sufficient demand. * PIM Manager: fix refining a search Due to not mapping the local index in the view to the parent’s index, refining only worked in views where parent and child had the same index for the contacts in the search view. * PIM Manager: fix starting when done via search When the unified address book (= FullView) was not running yet at the time when a client wanted to search it, the unified address book was not started and thus the search never returned results. * PIM Manager: fix writing contact, support photo and notes folks and EDS do not support writing properties in parallel (https://bugzilla.gnome.org/show_bug.cgi?id=652659). Must serialize setting of modified properties. * PIM Manager: fix incorrect contact removal signals in filtered view The filtered view did not check whether a parent’s removed contact was really part of the view before sending a removal signal for it. * D-Bus: missing out parameters in D-Bus introspection XML ([FDO #57292](https://bugs.freedesktop.org/show_bug.cgi?id=57292)) The problem was in the C++ D-Bus binding. If the method that gets bound to D-Bus returns a value, that value was ignored in the signature: int foo() => no out parameter It works when the method was declared as having a retval: void foo (int &result) => integer out parameter This problem existed for both the libdbus and the GIO D-Bus bindings. In SyncEvolution it affected methods like GetVersions(). * PIM Manager performance: pre-compute normalized telephone numbers Looking up by phone number spends most of its cycles in normalizing of the phone numbers in the unified address book. Instead of doing that work over and over again during the search, do it once while loading. Looking up a phone number only once does not gain from this change, it even gets slower (more memory intensive, less cache locality). Only searching multiple times becomes faster. Ultimately it would be best to store the normalized strings together with the telephone number inside EDS when the contact gets created. Work on that is in progress. * PIM Manager: improve performance of FullView sorting This fixes the hotspot during populating the FullView content: moving contacts around required copying IndividualData and thus copying complex C++ structs and strings. Storing pointers and moving those avoids that, with no lack of convenience thanks to boost::ptr_vector. Reordering also becomes faster, because the intermediate copy only needs to be of the pointers instead of the full content. * PIM Manager example: add benchmarking The new “checkpoints” split up the whole script run into pieces which are timed separately, with duration printed to stdout. In addition, tools like “perf” can be started for the duration of one phase. * EDS: fix creating databases –create-database was broken in combination with the final code in EDS 3.6 because it passed NULL for the UID to e_source_new_with_uid(), which is considered an error by the implementation of that method. Must use e_source_new() if we don’t have a UID. * fixed some memory leaks, extended tests to cover new features and bugs SyncEvolution 1.3.99.1, 25.10.2012 ================================== * workarounds for warnings from g++ 4.5 * engine: : local cache sync mode This patch introduces support for true one-way syncing (“caching”): the local datastore is meant to be an exact copy of the data on the remote side. The assumption is that no modifications are ever made locally outside of syncing. This is different from one-way sync modes, which allows local changes and only temporarily disables sending them to the remote side. Another goal of the new mode is to avoid data writes as much as possible. This new mode only works on the server side of a sync, where the engine has enough control over the data flow. Setting “sync” to: - “local-cache-incremental” will do an incremental sync (if possible) or a slow sync (otherwise). This is usually the right mode to use, and thus has “local-cache” as alias. - “local-cache-slow” will always do a slow sync. Useful for debugging or after (accidentally) making changes on the local side. An incremental sync will ignore such changes because they are not meant to happen, aren’t checked for to improve performance and thus will leave client and server out-of-sync! Both modes are recorded in the sync report of the local side. The target side is the client and records the normal “two-way” or “slow” sync modes. With the current SyncEvolution contact field list, first, middle and last name are used to find matches for contacts. For events, tasks and memos, time, summary and description are used. * HTTP proxy: useProxy=0 overrides http_* env variables Previously, if http_proxy was set, a proxy was used even if explicitly disabled. This prevented disabling the use of a proxy which only made sense in some cases, like accessing something that runs locally. Explicitly telling SyncEvolution to ignore http_proxy is necessary because it doesn’t support no_proxy. * WebDAV: auto-discovery fix With Google Contact + CardDAV the auto-discovery failed after finding the default address book, without reporting that result. * command line: implement –create/remove-database Creating a database is only possible with a chosen name. The UID is chosen automatically by the storage. Only implemented in the EDS backend. * file backend: sub-second mod time stamps Change tracking in the file backend used to be based on the modification time in seconds. When running many syncs quickly (as in testing), that can lead to changes not being detected when they happen within a second. Now the file backend also includes the sub-second part of the modification time stamp, if available. This change is relevant when upgrading SyncEvolution: most of the items will be considered “updated” once during the first sync after the upgrade (or a downgrade) because the revision strings get calculated differently. * D-Bus server: avoid progress outside of 0-100% range For example in the new TestLocalCache.testItemDelete100, the percentage value in the ProgressChanged signal become larger than 100 and then revert to 100 at the end of the sync. Seems the underlying calculation is faulty or simply inaccurate. This is not fixed. Instead the result is just clipped to the valid range. * code cleanup + improvements in testing Source, Installation, Further information ========================================= http://syncevolution.org/blogs/pohly/2013/syncevolution-13993-released Source code bundles for users are available in http://downloads.syncevolution.org/syncevolution/sources and the original source is [the git repositories](http://cgit.freedesktop.org/SyncEvolution/). i386, lpia and amd64 binaries for Debian-based distributions are available via the “unstable” syncevolution.org repository. Add the following entry to your /apt/source.list:
debhttp://downloads.syncevolution.org/aptunstablemain Then install “syncevolution-evolution”, “syncevolution-kde” and/or “syncevolution-activesync”. These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 10.04 LTS (Lucid), except for “syncevolution-activesync” which depends on libraries in Debian Squeeze, for example EDS 3.4. Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/). In contrast to 0.8.x archives, the 1.x .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps. More specific [HOWTOs](/wiki/howto) can be found in the Wiki.
Minor (or major, if you depend on auto syncing) bug fix release. Automatic syncing only ran once and notifications were not translated. * auto sync: only synced once ([FDO #56667](https://bugs.freedesktop.org/show_bug.cgi?id=56667)) A successful sync was incorrectly treated like a sync with a permanent failure, which prevents further automatic syncing. * auto sync: notifications were not translated The code which enabled localization of messages created by the D-Bus server was incomplete. Localization was only enabled accidentally through KDE if the KDE platform modules was enabled during compilation and installed. * HTTP Proxy: useProxy=0 overrides http_* env variables Previously, if http_proxy was set, a proxy was used even if explicitly disabled. This prevented disabling the use of a proxy which only made sense in some cases, like accessing something that runs locally. Explicitly telling SyncEvolution to ignore http_proxy is necessary because it doesn’t support no_proxy. * minor changes in testing and autotools files (missing Boost search path in gdbus* libs might have caused compile problems) Upgrading from release 1.2.x —————————- The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts): syncevolution –configure \ syncFormat=text/x-vcard \ mobical addressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually: syncevolution –configure \ enableRefreshSync=TRUE \ funambol Upgrading from releases before 1.2 ———————————- Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Source, Installation, Further information ========================================= Source code bundles for users are available in http://downloads.syncevolution.org/syncevolution/sources and the original source is in [the git repositories](http://cgit.freedesktop.org/SyncEvolution/). i386, lpia and amd64 binaries for Debian-based distributions are available via the “stable” syncevolution.org repository. Add the following entry to your /apt/source.list:
debhttp://downloads.syncevolution.org/aptstablemain Then install “syncevolution-evolution”, “syncevolution-kde” and/or “syncevolution-activesync”. These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy), except for “syncevolution-activesync” which depends on libraries in Debian Squeeze, for example EDS 3.4. Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/). In contrast to 0.8.x archives, the 1.x .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
SyncEvolution 1.3.1 fixes some minor issues found after 1.3 was released. * command line: fix output of –import for directories The running count at the start of the line (#0, #1, …) was not incremented when reading individual files from a directory. * Funambol: work around PHOTO TYPE=image/jpeg, part II The final version of the fix hadn’t made it into the source code. * vCalendar 1.0 + tasks: DUE date could be shifted by a day ([FDO #55238](https://bugs.freedesktop.org/show_bug.cgi?id=55238)) Because of incomplete support for time conversion, the due date could get mixed up when phone and PC were set to something other than UTC. Reported and fixed by Peter Jan. * syncevolution.org: syncevolution-evolution had incorrect dependencies Installation on older Linux distros was not possible because the ebook/ecal package dependencies were named incorrectly, for example libebook-1.2-10 instead of libebook1.2-10. Only more recent packages have the extra dash, for example libebook-1.2-12. Reported by Mariusz Sokolowski. * GTK-3 UI: fixed compile problem The GTK-3 UI depends on a class from gio-unix-2.0 and failed to compile on Fedora Core 16 because the configure checks for that lib (and thus the compiler flags) were missing. Reported by Peter Robinson. * Curl: allow using it in the D-Bus server In the past, using curl as HTTP transport in the syncevo-dbus-server was prevented, leading to “unsupported transport type is specified in the configuration”. The reason was that using curl would block the server and make it unresponsive on D-Bus. This reason has gone away, because now the HTTP traffic happens in a separate process. Thus now it is allowed to use curl in the syncevo-dbus-server. * fix for false negative in syncevo-dbus-server testing Upgrading from release 1.2.x —————————- The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts): syncevolution –configure \ syncFormat=text/x-vcard \ mobical addressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually: syncevolution –configure \ enableRefreshSync=TRUE \ funambol Upgrading from releases before 1.2 ———————————- Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Source, Installation, Further information ========================================= Source code bundles for users are available in http://downloads.syncevolution.org/syncevolution/sources and the original source is in [the git repositories](http://cgit.freedesktop.org/SyncEvolution/). i386, lpia and amd64 binaries for Debian-based distributions are available via the “stable” syncevolution.org repository. Add the following entry to your /apt/source.list:
debhttp://downloads.syncevolution.org/aptstablemain Then install “syncevolution-evolution”, “syncevolution-kde” and/or “syncevolution-activesync”. These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy), except for “syncevolution-activesync” which depends on libraries in Debian Squeeze, for example EDS 3.4. Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/). In contrast to 0.8.x archives, the 1.x .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
After almost three months of public beta testing the next major version of SyncEvolution is ready for release. The pre-releases did have the desired effect of flushing out bugs not found by nightly testing alone. Thanks everyone for packaging, downloading and testing them! Project status ============== SyncEvolution synchronizes personal information management (PIM) data via various protocols (SyncML, CalDAV/CardDAV, ActiveSync). It syncs contacts, appointments, tasks and memos. It syncs to web services or to SyncML-capable phones via Bluetooth. Binaries are available for Linux desktops (using GNOME Evolution, or KDE’s Akonadi), for MeeGo and for Maemo (Nokia N900, N9). The project moved its bug tracking from bugs.meego.com to bugs.freedesktop.org. All the old issues were migrated. Please file new issues here: http://bugs.freedesktop.org/enter_bug.cgi?product=SyncEvolution The git repositories were also moved to freedesktop.org: http://cgit.freedesktop.org/SyncEvolution/ SyncEvolution 1.3 ================= New features are KDE/Akonadi and ActiveSync support, not only in the source code but also in syncevolution.org binaries. ActiveSync is the recommended way of synchronizing contacts with Google: https://syncevolution.org/wiki/google-contacts-activesync The D-Bus server and local sync were rewritten considerably, to make the code cleaner and more robust. The CalDAV backend now also supports tasks and memos. CalDAV and CardDAV can be used in combination with a SyncML peer (“bridging SyncML and WebDAV”), thus allowing a device which only supports SyncML to talk to a WebDAV service without any intermediate storage. 1.3 contains bug fixes that were not backported to 1.2.x, so upgrading is recommended. For example, SyncEvolution 1.3 is required for Evolution 3.4, otherwise photos are not exported properly. Support for Evolution >= 3.6 is in the source code, but not in syncevolution.org binaries. Further workarounds for recent changes in Google CalDAV and Funambol One Media were added. Upgrading from release 1.2.x —————————- The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts): syncevolution –configure \ syncFormat=text/x-vcard \ mobical addressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually: syncevolution –configure \ enableRefreshSync=TRUE \ funambol Upgrading from releases before 1.2 ———————————- Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Changes 1.2.2 -> 1.3 ==================== * ActiveSync: updated to work with latest activesyncd and Google, package binaries Syncing Google contacts was added to the nightly testing. Syncing contacts and events with Exchange 2012 was already working. Setup instructions and known issues are described here: https://syncevolution.org/wiki/google-contacts-activesync * phone sync: delete<->delete conflict + phone calendar+todo sync ([BMC #23744](https://bugs.meego.com/show_bug.cgi?id=23744)) When deleting an item on phone and locally, the next sync failed with ERROR messages about “object not found”. This has several reasons: - libsynthesis super data store attempts to read items which may or may not exist (triggers ERROR message) - it checks for 404 but Evolution backends only return a generic database error (causes sync to fail) * phone sync: get phone vendor and model from Device ID profile ([BMC #736](https://bugs.meego.com/show_bug.cgi?id=736)) In the past we have relied on the user-modifiable device name to be the fingerprint for matching a phone to a template which is unreliable. This release changes this in the cases where the phone supports the Device ID profile (DIP). If support for DIP is detected, then we extract the vendor and product ids and attempt to associate them with a product and vendor name by using a newly added lookup table. This lookup table has to be maintained manually and depends on contributions by users to cover more devices. See http://blixtra.org/blog/2011/09/22/syncevolution-needs-you-or-at-least-your-bluetooth-phones/ * vCalendar 1.0: fixed recurring all-day event support vCalendar 1.0 cannot represent all-day events. The workarounds for mapping iCalendar 2.0 all-day events into vCalendar 1.0 was incomplete, leading to effects like shifting EXDATEs and end times. * Funambol: ignore UID Funambol’s OneMedia sends UID, but not RECURRENCE-ID. That becomes a problem when multiple events of the same event series are added to a backend which follows the iCalendar 2.0 standard (CalDAV, EDS, KDE), because these events all look like the master event, and there can be only one of those. SyncEvolution now strips the UID from all events coming from any Funambol server (regardless of the version). If a future Funambol server release adds support for both UID and RECURRENCE-ID, then SyncEvolution will have to be updated to take advantage of the improved server. Because the RECURRENCE-ID is also getting stripped (despite not being set at the moment), SyncEvolution should continue to work as it does now even if the server changes. It would have been nice to limit this workaround to affected Funambol server versions, but an inquiry on the Funambol mailing list didn’t get a reply, therefore SyncEvolution is playing it safe and assumes that all future Funambol releases will have the same problem. * Funambol: work around PHOTO TYPE=image/jpeg A combination of Funambol Android and Funambol server recently led to the Funambol server sending PHOTO data with TYPE=image/jpeg. This is invalid and caused EDS to reject the photo (Vladimir Elisseev, “[SyncEvolution] issues with syncing photos”). Work around the problem by only keeping the part of the type after the last slash, if there is any. For image/jpeg and similar types that leads to the desired value and does not affect valid values, because those do not contain a slash (http://www.iana.org/assignments/media-types/image/index.html). * Funambol: avoid slow syncs in refresh from server libsynthesis has traditionally implemented “refresh-from-server” as “delete local data” plus “slow” sync. This is more compatible, because some servers (like Google) do not support “refresh-from-server”. But it has the downside that the server cannot know that the client won’t send any data, and Funambol’s OneMedia now only allows one slow sync before blocking the next one for a certain period of time. This is done to prevent excessive resource usage by badly behaving clients. To accomodate both kinds of servers, the new “enableRefreshSync” sync property can be set set to explicitly allow the usage of the “refresh-from-server” sync mode. It’s off by default. The Funambol template has it turned on, existing configs must be updated manually (see upgrading comments below). * Mobical (aka Everdroid): stopped testing memo syncing Memos used to work, but now only trigger an unspecific 400 error on the server side. * GTK-UI: accept service config with a username again (BMC#23106) Suppressing configs with empty username had undesired side effects: modifying configs for direct syncing with a device incorrectly triggered the same error message, without any means of entering a username. The faulty check was removed without replacement. * GTK-UI: added GTK 3 version of UI When GTK 3 is found during compilation, a GTK 3 version of the UI is built. The source code of both is different to avoid excessive use of ifdefs. At the moment, both versions offer the same features. In the long run, the GTK 3 version will replace the GTK 2 version. * command line: added refresh/one-way-from-local/remote ([BMC #23537](https://bugs.meego.com/show_bug.cgi?id=23537)) The -from-client/server sync modes are confusing because the direction of the data exchange depends on which side acts as SyncML server or client. This release introduces new modes which use -from-local/remote instead. The statistics and messages also use these variants now. The old modes are still understood, but are declared as “not recommended” in the documentation. * command line: config and source names are optional ([BMC #23783](https://bugs.meego.com/show_bug.cgi?id=23783)) The need to add “foo” and “bar” pseudo config and source names to the command line even when all parameters for the operation where explicitly specified on the command line was confusing. Now it is possible to invoke item operations without the config and source name. Names which refer to non-existent configs are still accepted, as in previous releases. Typos are handled better by producing a detailed error report which includes (as applicable): - config doesn’t exist - source doesn’t exist or not selected - backend property not set Because luids used to be positional arguments after and , a new –luids keyword is necessary to indicate that the ensuing parameters are luids and not and . * command line: introduced –print-databases, supported for CalDAV/CardDAV Listing databases is now a dedicated operation, instead of being done whenever syncevolution was invoked without parameters. Advantages: - can be combined with property assignments for backends which do not work without that additional information, for example CalDAV/CardDAV: syncevolution –print-databases \ backend=[caldav|carddav] \ syncURL=… \ username=… \ password=… - can be done for configured sources * command line: use both stdout and stderr Traditionally, the “syncevolution” command line tool mixed its INFO/ERROR/DEBUG messages into the normal stdout. This has the major drawback that error messages get lost during operations like syncevolution –export - @default addressbook | grep “John Doe” Now anything which is not the expected result of the operation is always sent to stderr. Obviously this includes ERROR messages. INFO and DEBUG are harder to decide. Because they usually convey meta information about the running operation, they are also sent to stderr. The output of running a sync goes to both stdout (summary) and stderr (progress). * command line: allow setting empty properties Due to the way how properties were handled internally, it wasn’t possible to explicitly set a property to its default value. Instead the property was unset. For example, explicitly setting database= was not possible. This is necessary for client-test and ActiveSync, because client-test needs to know that the testing is expected to run with the default databases (something which normally is avoided by overwriting empty database properties). Now the “is set” state is tracked explicitly in the config storage and command line property APIs. Unsetting a property via the command line could be implemented with an explicit command line option, but is not supported at the moment. * command line: fixed –export When exporting items into a file, the delimiter between items was missing. * command line + local sync: fixed erroneous “Comparison impossible” output. “Comparison impossible” was incorrectly printed after a successful comparison on the target side of local sync. * local sync: fix timeout with local sync with libdbus When using libdbus instead of GIO D-Bus (as done by syncevolution.org binaries and SyncEvolution on Maemo), local sync may have aborted after 25 seconds when syncing many items with a D-Bus timeout error: [ERROR] sending message to child failed: org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible ca Reported by Toke Høiland-Jørgensen for Harmattan. Somehow not encountered elsewhere. * synccompare: shorter data dump of PHOTO A full comparison of the base64 PHOTO data can be very long. Now some key characteristics of the PHOTO data (number of characters in base64 encoding, number of bytes in decoded data, md5sum of decoded data) are printed instead. That way, unintended changes of the data (different encoding, different content) should still be found while testing and added/removed photos are nicely visible in synccompare diffs. * synccompare: fixed output for byte-identical duplicates If database dumps contained byte-identical duplicates, they were treated as a single item on the left side of a comparison. This caused erroneous “added” entries on the right side. * secure password storage: usage of GNOME Keyring vs. KDE KWallet configurable Automatically detecting KDE users is not possible at the moment. Instead KDE users have to manually set the new “keyring” global config property to “KDE” (case insensitive) if the SyncEvolution installation supports both, because GNOME Keyring is the default to avoid surprises for traditional users. If only KWallet support is enabled, then this is not necessary. “GNOME” and “true/false/1/0/yes/no” can also be set. This has the advantage that keyring usage can be enabled permanently for the command line in –daemon=no mode; normally keyrings are not used in that mode because accessing them can bring up UI dialogs. It also becomes possible to disable keyring usage in syncevo-dbus-server, something which couldn’t be done before. The –keyring command line option is still supported, as an alias for “[–sync-property] keyring=”. The default value for –keyring is true, to match the traditional behavior. In contrast to other sync properties, setting “keyring” does not require an explicit –run parameter. Again this is done to mirror traditional usage. * config: improved ‘maxlogdirs’ documentation The old explanation made it sound like nothing would get deleted by default (“If set, …”). That’s not correct, by default only 10 sessions are kept. Also explain the behavior of deleting intermediate sessions first. * Evolution: always create databases (PTCOM-113) Always try to create address book or calendar database, because even if there is a source there’s no guarantee that the actual database was created already; the original logic for only setting this when explicitly requesting a new database therefore failed in some cases. This problem affected users who had never created anything locally and wanted to use SyncEvolution to migrate their data. Now that works without having to create dummy entries first. * Evolution contacts: changed default sync format to vCard 3.0 vCard 3.0 is the better default because it has saner encoding rules and defines more properties, thus avoiding the need for non-standard extensions. However, Mobical has problems with the new default. See upgrade instructions below. * Evolution: added support for EDS 3.5.x When compiled against EDS 3.5.x or later, SyncEvolution now uses the backend code originally written for the EClient API introduced in EDS 3.2. That code was changed so that it works with the new include file rules and ESourceRegistry in EDS 3.5.x. Support for using the EClient API with EDS 3.4 was removed because maintaining three different flavors of the EDS backend code would be too much work and not gain much (just the possibility to test the EDSClient code with 3.4). At the moment, this is a compile time choice made automatically by configure. syncevolution.org binaries are compiled against an older EDS and thus do not work with EDS 3.5.x or later. EDS 3.5.x handles authentication itself, using a standard system prompt if necessary. SyncEvolution can no longer provide the password, and thus the “databaseUser/Password” options have no effect when using EDS 3.5.x. * D-Bus server: fixed HTTP presence for recent libdbus Testing with libdbus 1.6.0 on Debian Testing failed because the lib changed some behavior: instead of looking up the owner of a certain bus name immediately, it now does that when invoking a method. Therefore the check for “have connection” in SyncEvolution was too simplistic and missed the fact that both were not usable, causing the server to assume that HTTP was down while in reality it should have assumed it to be up. This prevented auto-syncing and manually clicking “Sync” in the GTK UI. * D-Bus server: made notification verbosity configurable with “notifyLevel” The new “notifyLevel” per-peer configuration option allows users to control how many desktop notifications the D-Bus server produces while executing an automatic sync: 0 - suppress all notifications 1 - show only errors 2 - show information about changes and errors (in practice currently the same as level 3) 3 - show all notifications, including starting a sync (default) * WebDAV: fixed data corruption issue when uploading item with long UID In some cases data with a very long UID wasn’t handled correctly, causing the out-going data to be malformed and probably causing a rejection by the server. The root cause is incorrect string handling. In releases before 1.2.99.1, only the –import operation of contacts into CardDAV were affected. In 1.2.99.1, the same code also got used for calendar items and then could also affect syncing. * CalDAV: updated Google workarounds Google started sending empty items (VCALENDAR with no VEVENT inside) which cannot be removed. SyncEvolution 1.3 ignores such items. The workaround for a 404 from Google Calendar for a GET (sending a REPORT request matching the item’s UID) was broken: first, processing the result ended up calling the unset responseEnd boost function pointer, which caused the request to fail. Second, getting multiple items wasn’t handled (data from all items concatenated together was used). That can happen in the somewhat unlike case that some items have a UID which is a complete superset of the requested UID - not realistic in real life, but happens during testing. * Google Calendar: updated URL redirect handling Google Calendar sometimes returns redirect requests to human-readable web sites (an “unavailable” page, a login form). This is of course bogus when the client is an automated CalDAV client. The “unavailable.html” case was already handled. Made it a bit more flexible to also catch possible variations of it (additional parameters, https instead of http). Added the https://accounts.google.com/ServiceLogin case. Not sure whether retrying will help in that case, but there’s not much else that SyncEvolution can do. * WebDAV: bridge with SyncML Now a peer accessed via SyncML can read/write data stored in a CalDAV/CardDAV server directly. This can be used to connect a device which only supports SyncML to a CalDAV/CardDAV server, or sync data between a SyncML server and a CalDAV/CardDAV server. See “CalDAV and CardDAV” in the README for details. * WebDAV: improved –configure Added INFO output about checking sources. This helps with WebDAV when the server cannot be contacted (dead, misconfigured) because otherwise there would be no indication at all why the –configure operation seems to hang. Here is some example output, including aborting:
$syncevolution--configure--templatewebdav\syncURL=http://192.168.1.100:9000/\username=foopassword=barretryDuration=2s\target-config@webdav-temp[INFO]creatingconfigurationtarget-config@webdav-temp[INFO]addressbook:lookingfordatabases...[INFO]addressbook:nodatabasetosynchronize[INFO]calendar:lookingfordatabases...[INFO]calendar:nodatabasetosynchronize[INFO]memo:lookingfordatabases...[INFO]memo:nodatabasetosynchronize[INFO]todo:lookingfordatabases...[INFO]todo:nodatabasetosynchronize It timed out fairly quickly here because of the retryDuration=2s. That also gets placed in the resulting config, which is probably not desired. Aborting the operation is now supported:
$syncevolution--configure\--templatewebdav\syncURL=http://192.168.1.100:9000/\username=foopassword=bar\target-config@webdav-temp[INFO]creatingconfigurationtarget-config@webdav-temp[INFO]addressbook:lookingfordatabases...^C[INFO]Askingtosuspend...[INFO]PressCTRL-Cagainquickly(within2s)tostopimmediately(cancauseproblemsinthefuture!)^C[INFO]Abortingimmediately...[ERROR]errorcodefromSyncEvolutionabortedonbehalfofuser(local,status20017):abortingasrequestedbyuser It would be good to make the CTRL-C handling code aware that it can abort immediately instead of doing the intermediate “asking to suspend” step, which only makes sense for sync sessions. * WebDAV: support tasks and memos ([BMC #24893](https://bugs.meego.com/show_bug.cgi?id=24893)) The new backend property values “CalDAVTodo” and “CalDAVJournal” select tasks resp. memos stored in a CalDAV collection. “CalDAV” continues to select events. Events, tasks and journals can be mixed in the same resource (= URL). However, this is less efficient than storing them separately. A good CalDAV server allows filtering items by type, and SyncEvolution uses that. However, it was found that Radicale 0.7 ignores this filtering, which could have led to data loss (SyncEvolution asks for all VTODOs in preparation for a “delete all items” operation in a “CalDAVTodo” source, gets also VJOURNALs, then deletes them). Therefore SyncEvolution plays it safe and downloads the VTODO and VJOURNAL data to double-check that it is working on the right items. This causes additional traffic for well-behaving servers; currently it cannot be turned off. Tasks are exchanged as vCalendar 1.0 or iCalendar 2.0 VJOURNAL. Memos are exchanged as VTODO or plain text. The logic for storing incoming plain text is slightly different compared to the way how the EDS memo backend did it: instead of copying the first line from the text into the summary, it is now moved. In other words, the first line gets stripped. The change is primarily technically motivated; both approaches have pros and cons. * WebDAV: improved Radicale support Radicale > 0.7 will return status 200 for delete requests; is now treated like 204 by SyncEvolution. 412 ‘Preconditiona Failed’ when asking to delete an already removed item is treated like the more common 404 ‘not found’. Same with 410 ‘gone’ instead of 404 when trying to read a non-existent item. * CalDAV/CardDAV sync: improved target side output Added a “target side of local sync ready” INFO message to introduce the output which has the target context in the [INFO] tag. The sync report from the target side now has the target context embedded in brackets after the “Changes applied during synchronization” header, to avoid ambiguities. Sometimes the backend has to resend requests because of temporary issues. If the problem turned out to be permanent, there was a long period of time, retryDuration=5 minutes to be precice, in which no visible progress happened. Now SyncEvolution’s WebDAV backend will print a message like this before going to sleep until it is time to retry: [INFO @googlecalendar] operation temporarily (?) failed, going to retry in 5.0s before giving up in 18.4s: PROPFIND: Neon error code 1: 401 Unauthorized The uncertainty comes from several factors. In this example, the 401 might indicate a permanent problem (wrong credentials), or it could be Google reporting a temporary authorization problem which is (probably) meant to slow down the client while it asks the user to re-enter the password. SyncEvolution only asks for passwords once, so it tries again with the same password if it was successful with it in the past. Otherwise it gives up immediately. Another dubious example are name server lookup errors. They can be permanent (wrong host name) or temporary (name server down). SyncEvolution errs on the side of retrying, to avoid interrupting an operation which still has a chance to continue. Output from the target side of a local sync was passed through stderr redirection as chunks of text to the frontends. This had several drawbacks: - forwarding only happened when the local sync parent was processing the output redirection, which (due to limitations of the implementation) only happens when it needs to print something itself - debug messages were not forwarded - message boundaries might have been lost In particular the new INFO messages are relevant while the sync runs and need to be shown immediately. * WebDAV: –status for WebDAV source aborted The command line –status operation did not complete when applied to a CalDAV/CardDAV source. Instead it aborted because the operation took a code path where the backend was not fully initialized. * file backend: more flexible sync support for memos The databaseFormat=text/calendar for memos did not support synchronizing as plain text. When using the new databaseFormat=text/calendar+plain, vCalendar/iCalendar/plain text are all valid sync formats; the storage is iCalendar 2.0 VJOURNAL in all cases. * WebDAV: avoid potential crash during database detection When a server responds to a PROPFIND for a path with results for some other path, then SyncEvolution crashed during the search for the default calendar or address book because of a bug in the code which was meant to handle that kind of response. Apparently Yahoo Calendar did that. Now seen again in combination with Radicale 0.6.4. In general, the code was made more robust to cope with bugs in Radicale 0.6.4. Later Radicale versions fixed these issues and also worked with SyncEvolution 1.2.2 without client-side workarounds. * WebDAV: better path normalization “syncURL” and “database” properties had to end in a trailing slash, otherwise items were not found (404 errors). Now the necessary slash is added automatically. * Curl transport: support SSLServerCertificates= When the setting refers to a directory, then CURLOPT_CAINFO doesn’t work (must be a file). Check this and use CURLOPT_CAPATH instead. Caveat: there are some comments in the API documentation about “NSS enabled libcurl” which supports a directory in CURLOPT_CAINFO. Hopefully providing an explicit path in CURLOPT_CAPATH also works in that configuration. * code cleanup + rewrite: syncing done in separate process syncevo-dbus-server now runs syncing in a separate process. Local sync also uses a second helper process. This makes the D-Bus server more responsive via D-Bus (no more blocking operations) and minimizes the effect of bugs in code involved with syncing (backends, system libraries, etc.). In the long term this restructuring will also allow more advanced features, like monitoring local or remote storage for changes. * SyncEvolution <-> SyncEvolution sync: multiple cycles per session SyncML only allows one send/receive cycle per session. There are cases (for example, client side merges data that a dumber server failed to match correctly) where client and server are still out of sync at the end of a cycle. When SyncEvolution syncs with another SyncEvolution instance (locally or remotely), both sides detect that the peer can continue syncing in the same session and start over automatically when needed. Previously the user had to start another sync session manually. To the user this is shown as “number of cycles” in a sync session in the sync report. “Restart” is the process of entering a new cycle. The cycles are also visible in the command line output as multiple INFO lines:
`
[INFO] eds_contact: starting first time sync from client (peer is server)
[INFO] creating complete data backup of source eds_contact before sync (enabled with dumpData and needed for prin
Local data changes to be applied during synchronization:
*** eds_contact ***
no changes
Work towards the 1.3 release of SyncEvolution continued with two more snapshots since the last announcement. The project also moved its bug tracking from bugs.meego.com to bugs.freedesktop.org. Most of the old issues were already migrated, the rest will follow later. Please [file new issues](http://bugs.freedesktop.org/enter_bug.cgi?product=SyncEvolution) there. The git repos are still at meego.com; they will move to freedesktop.org eventually. SyncEvolution 1.2.99.3 -> 1.2.99.4, 07.08.2012 ============================================== Another release candidate for SyncEvolution 1.3. Lesson learned: declaring a snapshot as “final” is a good way of luring the hidden bugs into the light. Of course, then another snapshot is needed… Details: * D-Bus server: fix support for autoSyncDelay > 0 Auto syncing was not getting triggered when using an autoSyncDelay > 0; by default it is 5 minutes. Thanks to Vladimir Elisseev for reporting this problem. * command line: fixed –export When exporting items into a file, the delimiter between items was missing. * config: improved ‘maxlogdirs’ documentation The old explanation made it sound like nothing would get deleted by default (“If set, …”). That’s not correct, by default only 10 sessions are kept. Also explain the behavior of deleting intermediate sessions first. * developers: fixed D-Bus interface XML Reverted to Qt 4.x compatible annotations and changed “templateName” to “getTemplate” to make it more obvious what the parameter does. Only relevant for the out-of-tree Qt UI. Fixed accidental removal of the “template” parameter in Session.GetNamedConfig(). Was not used in practice, but has to be correct in case that someone wants to use it. SyncEvolution 1.2.99.2 -> 1.2.99.3, 24.07.2012 ============================================== Final release candidate for SyncEvolution 1.3 - fingers crossed, knock on wood, etc. ActiveSync is now available in binaries from syncevolution.org and becomes the recommended way of synchronizing contacts with Google. EDS 3.5.x and later are supported when compiling from source; syncevolution.org binaries continue to support only EDS up to 3.4. Details: * EDS: added support for EDS 3.5.x When compiled against EDS 3.5.x or later, SyncEvolution now uses the backend code originally written for the EClient API introduced in EDS 3.2. That code was changed so that it works with the new include file rules and ESourceRegistry in EDS 3.5.x. Support for using the EClient API with EDS 3.4 was removed because maintaining three different flavors of the EDS backend code would be too much work and not gain much (just the possibility to test the EDSClient code with 3.4). At the moment, this is a compile time choice made automatically by configure. syncevolution.org binaries are compiled against an older EDS and thus do not work with EDS 3.5.x or later. EDS 3.5.x handles authentication itself, using a standard system prompt if necessary. SyncEvolution can no longer provide the password, and thus the “databaseUser/Password” options have no effect when using EDS 3.5.x. * ActiveSync: updated to work with latest activesyncd and Google, package binaries Syncing Google contacts was added to the nightly testing. Syncing contacts and events with Exchange 2012 was already working. Setup instructions and known issues are described here: https://syncevolution.org/wiki/google-contacts-activesync * local sync: don’t drop data comparison output on target side synccompare on the target side of a local sync was invoked with its output being redirected via an unreliable socket to the local sync parent. When the output was large, some of it might have been lost. * local sync: fixed crash When processing stdout from syncevo-local-child in syncevo-dbus-helper, the LogRedirect class was invoked recursively and tried to print the same stdout data repeatedly until the syncevo-dbus-helper crashed due to the infinite recurssion. * local sync: fixed helper process shutdown in case of parent failure The helper process only detected that the parent failed when it tried to log something while the parent had already shut down the D-Bus connection. Even that did not work reliably and differed between D-Bus libdbus and GIO. Added several test cases and fixes for “process died prematurely” error scenarios. * Mobical (aka Everdroid): stopped testing memo syncing Memos used to work, but now only trigger an unspecific 400 error on the server side. * autotools: ensure that link lines are complete As mentioned by Tino Keitel on the mailing list, some libs and executables were only implicitly linked against libraries that they called directly. This happened to work by chance because these libraries ended up in the running executable anyway, due to indirect loading. Now there is a “make installcheck” test for this kind of defect and the makefiles were updated to avoid it. One exception is libsmltk, which depends on the caller providing SySync logging support. * D-Bus server: fixed HTTP presence for recent libdbus Testing with libdbus 1.6.0 on Debian Testing failed because the lib changed some behavior: instead of looking up the owner of a certain bus name immediately, it now does that when invoking a method. Therefore the check for “have connection” in SyncEvolution was too simplistic and missed the fact that both were not usable, causing the server to assume that HTTP was down while in reality it should have assumed it to be up. This prevented auto-syncing and manually clicking “Sync” in the GTK UI. * syncevolution.org: declare dependencies on libical and EDS Let the bundle .deb depend on libical if the lib was enabled during compilation (for example, for CalDAV). This ensures that it gets installed on systems which otherwise don’t have it. “syncevolution-evolution” is compatible (and depends on) EDS up to and including 3.4. The package now declares that dependency and conflicts with more recent EDS, because even if the older EDS libs are still installed they won’t work when the rest of EDS was updated. * CalDAV + syncevolution.org: fixed segfault without libical+libecal When libical and libecal were not installed, trying to use the CalDAV backend for VEVENTs segfaulted because it depends on libical and did not check properly for it. Only affected syncevolution.org binaries. Upgrading from release 1.2.x: The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts): syncevolution –configure \ syncFormat=text/x-vcard \ mobical addressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually: syncevolution –configure \ enableRefreshSync=TRUE \ funambol Upgrading from releases before 1.2: Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Source, Installation, Further information =============================== Source snapshots are in http://downloads.syncevolution.org/syncevolution/sources i386, lpia and amd64 binaries for Debian-based distributions are available via the “unstable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:
debhttp://downloads.syncevolution.org/aptunstablemain These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/evolution). In contrast to 0.8.x archives, the new .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
Final release candidate for SyncEvolution 1.3 - fingers crossed, knock on wood, etc. ActiveSync is now available in binaries from syncevolution.org and becomes the recommended way of [synchronizing contacts with Google](/wiki/google-contacts-activesync). EDS 3.5.x and later are supported when compiling from source; syncevolution.org binaries continue to support only EDS up to 3.4. Details: * EDS: added support for EDS 3.5.x When compiled against EDS 3.5.x or later, SyncEvolution now uses the backend code originally written for the EClient API introduced in EDS 3.2. That code was changed so that it works with the new include file rules and ESourceRegistry in EDS 3.5.x. Support for using the EClient API with EDS 3.4 was removed because maintaining three different flavors of the EDS backend code would be too much work and not gain much (just the possibility to test the EDSClient code with 3.4). At the moment, this is a compile time choice made automatically by configure. syncevolution.org binaries are compiled against an older EDS and thus do not work with EDS 3.5.x or later. EDS 3.5.x handles authentication itself, using a standard system prompt if necessary. SyncEvolution can no longer provide the password, and thus the “databaseUser/Password” options have no effect when using EDS 3.5.x. * ActiveSync: updated to work with latest activesyncd and Google, package binaries Syncing Google contacts was added to the nightly testing. Syncing contacts and events with Exchange 2012 was already working. Setup instructions and known issues are described here: https://syncevolution.org/wiki/google-contacts-activesync * local sync: don’t drop data comparison output on target side synccompare on the target side of a local sync was invoked with its output being redirected via an unreliable socket to the local sync parent. When the output was large, some of it might have been lost. * local sync: fixed crash When processing stdout from syncevo-local-child in syncevo-dbus-helper, the LogRedirect class was invoked recursively and tried to print the same stdout data repeatedly until the syncevo-dbus-helper crashed due to the infinite recurssion. * local sync: fixed helper process shutdown in case of parent failure The helper process only detected that the parent failed when it tried to log something while the parent had already shut down the D-Bus connection. Even that did not work reliably and differed between D-Bus libdbus and GIO. Added several test cases and fixes for “process died prematurely” error scenarios. * Mobical (aka Everdroid): stopped testing memo syncing Memos used to work, but now only trigger an unspecific 400 error on the server side. * autotools: ensure that link lines are complete As mentioned by Tino Keitel on the mailing list, some libs and executables were only implicitly linked against libraries that they called directly. This happened to work by chance because these libraries ended up in the running executable anyway, due to indirect loading. Now there is a “make installcheck” test for this kind of defect and the makefiles were updated to avoid it. One exception is libsmltk, which depends on the caller providing SySync logging support. * D-Bus server: fixed HTTP presence for recent libdbus Testing with libdbus 1.6.0 on Debian Testing failed because the lib changed some behavior: instead of looking up the owner of a certain bus name immediately, it now does that when invoking a method. Therefore the check for “have connection” in SyncEvolution was too simplistic and missed the fact that both were not usable, causing the server to assume that HTTP was down while in reality it should have assumed it to be up. This prevented auto-syncing and manually clicking “Sync” in the GTK UI. * syncevolution.org: declare dependencies on libical and EDS Let the bundle .deb depend on libical if the lib was enabled during compilation (for example, for CalDAV). This ensures that it gets installed on systems which otherwise don’t have it. “syncevolution-evolution” is compatible (and depends on) EDS up to and including 3.4. The package now declares that dependency and conflicts with more recent EDS, because even if the older EDS libs are still installed they won’t work when the rest of EDS was updated. * CalDAV + syncevolution.org: fixed segfault without libical+libecal When libical and libecal were not installed, trying to use the CalDAV backend for VEVENTs segfaulted because it depends on libical and did not check properly for it. Only affected syncevolution.org binaries. Upgrading from release 1.2.x: ——————————————- The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts):
syncevolution--configure\syncFormat=text/x-vcard\mobicaladdressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually:
syncevolution--configure\enableRefreshSync=TRUE\funambol Upgrading from releases before 1.2: ————————————————— Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Source, Installation, Further information =============================== Source snapshots are in http://downloads.syncevolution.org/syncevolution/sources i386, lpia and amd64 binaries for Debian-based distributions are available via the “unstable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:
debhttp://downloads.syncevolution.org/aptunstablemain These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/evolution). In contrast to 0.8.x archives, the new .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
Next step towards SyncEvolution 1.3. It adds a workaround for Funambol’s OneMedia and fixes an old bug which became more severe in 1.2.99.1. Also has some usability improvements for CalDAV/CardDAV. Hopefully it will not take long to stabilize the code, so test it now while it is still hot :-) Details: * Funambol: ignore UID Funambol’s OneMedia sends UID, but not RECURRENCE-ID. That becomes a problem when multiple events of the same event series are added to a backend which follows the iCalendar 2.0 standard (CalDAV, EDS, KDE), because these events all look like the master event, and there can be only one of those. SyncEvolution now strips the UID from all events coming from any Funambol server (regardless of the version). If a future Funambol server release adds support for both UID and RECURRENCE-ID, then SyncEvolution will have to be updated to take advantage of the improved server. Because the RECURRENCE-ID is also getting stripped (despite not being set at the moment), SyncEvolution should continue to work as it does now even if the server changes. It would have been nice to limit this workaround to affected Funambol server versions, but an inquiry on the Funambol mailing list didn’t get a reply, therefore SyncEvolution is playing it safe and assumes that all future Funambol releases will have the same problem. * WebDAV: fixed data corruption issue when uploading item with long UID In some cases data with a very long UID wasn’t handled correctly, causing the out-going data to be malformed and probably causing a rejection by the server. The root cause is incorrect string handling. In releases before 1.2.99.1, only the –import operation of contacts into CardDAV were affected. In 1.2.99.1, the same code also got used for calendar items and then could also affect syncing. * engine: add DTSTAMP+LAST-MODIFIED before writing calendar items When writing calendar items into a backend storage as iCalendar 2.0 or vCalendar 1.0, they should have DTSTAMP and LAST-MODIFIED values. DTSTAMP is expected by some CalDAV servers (like Apple). LAST-MODIFIED is usually added by the storage, but not always. In the text/plain -> syncevolution -> text/calendar -> Radicale -> EDS -> syncevolution chain the LAST-MODIFIED was not added by Radicale, which caused problems for change tracking in an EDS-based SyncEvolution. Also necessary when importing from a phone using vCalendar without DTSTAMP directly into CalDAV. * Google Calendar: updated URL redirect handling Google Calendar sometimes returns redirect requests to human-readable web sites (an “unavailable” page, a login form). This is of course bogus when the client is an automated CalDAV client. The “unavailable.html” case was already handled. Made it a bit more flexible to also catch possible variations of it (additional parameters, https instead of http). Added the https://accounts.google.com/ServiceLogin case. Not sure whether retrying will help in that case, but there’s not much else that SyncEvolution can do. * CalDAV + VJOURNAL: handle UID conflicts When asked to insert a VJOURNAL which already existed (= same UID), CalDAV servers respond with a 412 “Precondition failed” error. This needs to be detected and translated into an “item needs to be merged” result so that the engine can load the existing item, merge the data, and then write back. * WebDAV: –status for WebDAV source aborted The command line –status operation did not complete when applied to a CalDAV/CardDAV source. Instead it aborted because the operation took a code path where the backend was not fully initialized. * CalDAV/CardDAV sync: improved target side output Added a “target side of local sync ready” INFO message to introduce the output which has the target context in the [INFO] tag. The sync report from the target side now has the target context embedded in brackets after the “Changes applied during synchronization” header, to avoid ambiguities. Sometimes the backend has to resend requests because of temporary issues. If the problem turned out to be permanent, there was a long period of time, retryDuration=5 minutes to be precice, in which no visible progress happened. Now SyncEvolution’s WebDAV backend will print a message like this before going to sleep until it is time to retry: [INFO @googlecalendar] operation temporarily (?) failed, going to retry in 5.0s before giving up in 18.4s: PROPFIND: Neon error code 1: 401 Unauthorized The uncertainty comes from several factors. In this example, the 401 might indicate a permanent problem (wrong credentials), or it could be Google reporting a temporary authorization problem which is (probably) meant to slow down the client while it asks the user to re-enter the password. SyncEvolution only asks for passwords once, so it tries again with the same password if it was successful with it in the past. Otherwise it gives up immediately. Another dubious example are name server lookup errors. They can be permanent (wrong host name) or temporary (name server down). SyncEvolution errs on the side of retrying, to avoid interrupting an operation which still has a chance to continue. Output from the target side of a local sync was passed through stderr redirection as chunks of text to the frontends. This had several drawbacks: - forwarding only happened when the local sync parent was processing the output redirection, which (due to limitations of the implementation) only happens when it needs to print something itself - debug messages were not forwarded - message boundaries might have been lost In particular the new INFO messages are relevant while the sync runs and need to be shown immediately. * command line: fixed password + property lookup during –print-databases –print-databases for an existing configuration did not look up passwords stored in a keyring, causing the operation to fail for backends like CalDAV/CardDAV where credentials are required. Overriding source properties in that case also only worked when using the unqualified property name (“databasePassword=foo”) but not when using the source name as prefix (“calendar/databasePassword=foo”). * Developers: Implementing the improved local sync output required extending the D-Bus API. The Server.LogOutput signal now has an additional “process name” parameter. Normally it is empty. For messages originating from the target side, it carries that extra target context string. This D-Bus API change is backward compatible. Older clients can still subscribe to and decode the LogOutput messages, they’ll simply ignore the extra parameter. Newer clients expecting that extra parameter won’t work with an older D-Bus daemon: they’ll fail to decode the D-Bus message. Upgrading from release 1.2.x: ——————————————- The sync format of existing configurations for Mobical (aka Everdroid) must be updated manually, because the server has encoding problems when using vCard 3.0 (now the default for Evolution contacts):
syncevolution--configure\syncFormat=text/x-vcard\mobicaladdressbook The Funambol template explicitly enables usage of the “refresh-from-server” sync mode to avoid getting throttled with 417 ‘retry later’ errors. The same must be added to existing configs manually:
syncevolution--configure\enableRefreshSync=TRUE\funambol Upgrading from releases before 1.2: ————————————————— Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Releases >= 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Source, Installation, Further information =============================== Source snapshots are in http://downloads.syncevolution.org/syncevolution/sources i386, lpia and amd64 binaries for Debian-based distributions are available via the “unstable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:
debhttp://downloads.syncevolution.org/aptunstablemain These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/evolution). In contrast to 0.8.x archives, the new .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
First pre-release of SyncEvolution 1.3. Contains bug fixes that were not backported to 1.2.x, so upgrading is recommended. For example, SyncEvolution 1.3 is required for Evolution 3.4, otherwise photos are not exported properly. Further workarounds for recent changes in Google CalDAV were added. Major new features are KDE/Akonadi support in the syncevolution.org binaries and ActiveSync support (only in the source code). The D-Bus server and local sync were rewritten considerably, to make the code cleaner and more robust. The CalDAV backend now also supports tasks and memos. Details: * phone sync: delete<->delete conflict + phone calendar+todo sync ([BMC #23744](https://bugs.meego.com/show_bug.cgi?id=23744)) When deleting an item on phone and locally, the next sync failed with ERROR messages about “object not found”. This has several reasons: - libsynthesis super data store attempts to read items which may or may not exist (triggers ERROR message) - it checks for 404 but Evolution backends only return a generic database error (causes sync to fail) * phone sync: get phone vendor and model from Device ID profile ([BMC #736](https://bugs.meego.com/show_bug.cgi?id=736)) In the past we have relied on the user-modifiable device name to be the fingerprint for matching a phone to a template which is unreliable. This release changes this in the cases where the phone supports the Device ID profile (DIP). If support for DIP is detected, then we extract the vendor and product ids and attempt to associate them with a product and vendor name by using a newly added lookup table. This lookup table has to be maintained manually and depends on contributions by users to cover more devices. See http://blixtra.org/blog/2011/09/22/syncevolution-needs-you-or-at-least-your-bluetooth-phones/ * vCalendar 1.0: fixed recurring all-day event support vCalendar 1.0 cannot represent all-day events. The workarounds for mapping iCalendar 2.0 all-day events into vCalendar 1.0 was incomplete, leading to effects like shifting EXDATEs and end times. * GTK-UI: accept service config with a username again (BMC#23106) Suppressing configs with empty username had undesired side effects: modifying configs for direct syncing with a device incorrectly triggered the same error message, without any means of entering a username. The faulty check was removed without replacement. * GTK-UI: added GTK 3 version of UI When GTK 3 is found during compilation, a GTK 3 version of the UI is built. The source code of both is different to avoid excessive use of ifdefs. At the moment, both versions offer the same features. In the long run, the GTK 3 version will replace the GTK 2 version. * command line: added refresh/one-way-from-local/remote ([BMC #23537](https://bugs.meego.com/show_bug.cgi?id=23537)) The -from-client/server sync modes are confusing because the direction of the data exchange depends on which side acts as SyncML server or client. This release introduces new modes which use -from-local/remote instead. The statistics and messages also use these variants now. The old modes are still understood, but are declared as “not recommended” in the documentation. * command line: config and source names are optional ([BMC #23783](https://bugs.meego.com/show_bug.cgi?id=23783)) The need to add “foo” and “bar” pseudo config and source names to the command line even when all parameters for the operation where explicitly specified on the command line was confusing. Now it is possible to invoke item operations without the config and source name. Names which refer to non-existent configs are still accepted, as in previous releases. Typos are handled better by producing a detailed error report which includes (as applicable): - config doesn’t exist - source doesn’t exist or not selected - backend property not set Because luids used to be positional arguments after and , a new –luids keyword is necessary to indicate that the ensuing parameters are luids and not and . * command line: introduced –print-databases, supported for CalDAV/CardDAV Listing databases is now a dedicated operation, instead of being done whenever syncevolution was invoked without parameters. Advantages: - can be combined with property assignments for backends which do not work without that additional information, for example CalDAV/CardDAV:
-canbedoneforconfiguredsources\*commandline:usebothstdoutandstderrTraditionally,the"syncevolution"commandlinetoolmixeditsINFO/ERROR/DEBUGmessagesintothenormalstdout.Thishasthemajordrawbackthaterrormessagesgetlostduringoperationslikesyncevolution--export-@defaultaddressbook|grep"JohnDoe"Nowanythingwhichnottheexpectedresultoftheoperationisalwayssenttostderr.ObviouslythisincludesERRORmessages.INFOandDEBUGarehardertodecide.Becausetheyusuallyconveymetainformationabouttherunningoperation,theyarealsosenttostderr.Theoutputofrunningasyncgoestobothstdout(summary)andstderr(progress).\*commandline:allowsettingemptypropertiesDuetothewayhowpropertieswerehandledinternally,itwasn'tpossibletoexplicitlysetapropertytoitsdefaultvalue.Insteadthepropertywasunset.Forexample,explicitlysettingdatabase=wasnotpossible.Thisisnecessaryforclient-testandActiveSync,becauseclient-testneedstoknowthatthetestingisexpectedtorunwiththedefaultdatabases(somethingwhichnormallyisavoidedbyoverwritingemptydatabaseproperties).Nowthe"isset"stateistrackedexplicitlyintheconfigstorageandcommandlinepropertyAPIs.Unsettingapropertyviathecommandlinecouldbeimplementedwithanexplicitcommandlineoption,butisnotsupportedatthemoment.\*commandline+localsync:fixederroneous"Comparisonimpossible"output."Comparisonimpossible"wasincorrectlyprintedafterasuccessfulcomparisononthetargetsideoflocalsync.\*synccompare:shorterdatadumpofPHOTOAfullcomparisonofthebase64PHOTOdatacanbeverylong.NowsomekeycharacteristicsofthePHOTOdata(numberofcharactersinbase64encoding,numberofbytesindecodeddata,md5sumofdecodeddata)areprintedinstead.Thatway,unintendedchangesofthedata(differentencoding,differentcontent)shouldstillbefoundwhiletestingandadded/removedphotosarenicelyvisibleinsynccomparediffs.\*synccompare:fixedoutputforbyte-identicalduplicatesIfdatabasedumpscontainedbyte-identicalduplicates,theyweretreatedasasingleitemontheleftsideofacomparison.Thiscausederroneous"added"entriesontherightside.\*securepasswordstorage:usageofGNOMEKeyringvs.KDEKWalletconfigurableAutomaticallydetectingKDEusersisnotpossibleatthemoment.InsteadKDEusershavetomanuallysetthenew"keyring"globalconfigpropertyto"KDE"(caseinsensitive)iftheSyncEvolutioninstallationsupportsboth,becauseGNOMEKeyringisthedefaulttoavoidsurprisesfortraditionalusers.IfonlyKWalletsupportisenabled,thenthisisnotnecessary."GNOME"and"true/false/1/0/yes/no"canalsobeset.Thishastheadvantagethatkeyringusagecanbeenabledpermanentlyforthecommandlinein--daemon=nomode;normallykeyringsarenotusedinthatmodebecauseaccessingthemcanbringupUIdialogs.Italsobecomespossibletodisablekeyringusageinsyncevo-dbus-server,somethingwhichcouldn'tbedonebefore.The--keyringcommandlineoptionisstillsupported,asanaliasfor"[--sync-property]keyring=<value>".Thedefaultvaluefor--keyringistrue,tomatchthetraditionalbehavior.Incontrasttoothersyncproperties,setting"keyring"doesnotrequireanexplicit--runparameter.Againthisisdonetomirrortraditionalusage.\*Evolution:alwayscreatedatabases(PTCOM-113)Alwaystrytocreateaddressbookorcalendardatabase,becauseevenifthereisasourcethere'snoguaranteethattheactualdatabasewascreatedalready;theoriginallogicforonlysettingthiswhenexplicitlyrequestinganewdatabasethereforefailedinsomecases.ThisproblemaffecteduserswhohadnevercreatedanythinglocallyandwantedtouseSyncEvolutiontomigratetheirdata.Nowthatworkswithouthavingtocreatedummyentriesfirst.\*Evolutioncontacts:changeddefaultsyncformattovCard3.0vCard3.0isthebetterdefaultbecauseithassanerencodingrulesanddefinesmoreproperties,thusavoidingtheneedfornon-standardextensions.However,Mobicalhasproblemswiththenewdefault.Seeupgradeinstructionsbelow.\*D-Busserver:madenotificationverbosityconfigurablewith"notifyLevel"Thenew"notifyLevel"per-peerconfigurationoptionallowsuserstocontrolhowmanydesktopnotificationstheD-Busserverproduceswhileexecutinganautomaticsync:0-suppressallnotifications1-showonlyerrors2-showinformationaboutchangesanderrors(inpracticecurrentlythesameaslevel3)3-showallnotifications,includingstartingasync(default)\*CalDAV:updatedGoogleworkaroundsGooglestartedsendingemptyitems(VCALENDARwithnoVEVENTinside)whichcannotberemoved.SyncEvolution1.3ignoressuchitems.Theworkaroundfora404fromGoogleCalendarforaGET(sendingaREPORTrequestmatchingtheitem'sUID)wasbroken:first,processingtheresultendedupcallingtheunsetresponseEndboostfunctionpointer,whichcausedtherequesttofail.Second,gettingmultipleitemswasn'thandled(datafromallitemsconcatenatedtogetherwasused).ThatcanhappeninthesomewhatunlikecasethatsomeitemshaveaUIDwhichisacompletesupersetoftherequestedUID-notrealisticinreallife,buthappensduringtesting.\*WebDAV:bridgewithSyncMLNowapeeraccessedviaSyncMLcanread/writedatastoredinaCalDAV/CardDAVserverdirectly.ThiscanbeusedtoconnectadevicewhichonlysupportsSyncMLtoaCalDAV/CardDAVserver,orsyncdatabetweenaSyncMLserverandaCalDAV/CardDAVserver.See"CalDAVandCardDAV"intheREADMEfordetails.\*WebDAV:improved--configureAddedINFOoutputaboutcheckingsources.ThishelpswithWebDAVwhentheservercannotbecontacted(dead,misconfigured)becauseotherwisetherewouldbenoindicationatallwhythe--configureoperationseemstohang.Hereissomeexampleoutput,includingaborting:
$ syncevolution –configure –template webdav
syncURL=http://192.168.1.100:9000/
username=foo password=bar retryDuration=2s
target-config@webdav-temp
[INFO] creating configuration target-config@webdav-temp
[INFO] addressbook: looking for databases…
[INFO] addressbook: no database to synchronize
[INFO] calendar: looking for databases…
[INFO] calendar: no database to synchronize
[INFO] memo: looking for databases…
[INFO] memo: no database to synchronize
[INFO] todo: looking for databases…
[INFO] todo: no database to synchronize
IttimedoutfairlyquicklyherebecauseoftheretryDuration=2s.Thatalsogetsplacedintheresultingconfig,whichisprobablynotdesired.Abortingtheoperationisnowsupported:
$ syncevolution –configure
–template webdav
syncURL=http://192.168.1.100:9000/
username=foo password=bar
target-config@webdav-temp
[INFO] creating configuration target-config@webdav-temp
[INFO] addressbook: looking for databases…
^C[INFO] Asking to suspend…
[INFO] Press CTRL-C again quickly (within 2s) to stop immediately (can cause problems in the future!)
^C[INFO] Aborting immediately …
[ERROR] error code from SyncEvolution aborted on behalf of user (local, status 20017): aborting as requested by user
ItwouldbegoodtomaketheCTRL-Chandlingcodeawarethatitcanabortimmediatelyinsteadofdoingtheintermediate"askingtosuspend"step,whichonlymakessenseforsyncsessions.\*WebDAV:supporttasksandmemos(\[BMC#24893\](https://bugs.meego.com/show_bug.cgi?id=24893))Thenewbackendpropertyvalues"CalDAVTodo"and"CalDAVJournal"selecttasksresp.memosstoredinaCalDAVcollection."CalDAV"continuestoselectevents.Events,tasksandjournalscanbemixedinthesameresource(=URL).However,thisislessefficientthanstoringthemseparately.AgoodCalDAVserverallowsfilteringitemsbytype,andSyncEvolutionusesthat.However,itwasfoundthatRadicale0.7ignoresthisfiltering,whichcouldhaveledtodataloss(SyncEvolutionasksforallVTODOsinpreparationfora"deleteallitems"operationina"CalDAVTodo"source,getsalsoVJOURNALs,thendeletesthem).ThereforeSyncEvolutionplaysitsafeanddownloadstheVTODOandVJOURNALdatatodouble-checkthatitisworkingontherightitems.Thiscausesadditionaltrafficforwell-behavingservers;currentlyitcannotbeturnedoff.TasksareexchangedasvCalendar1.0oriCalendar2.0VJOURNAL.MemosareexchangedasVTODOorplaintext.ThelogicforstoringincomingplaintextisslightlydifferentcomparedtothewayhowtheEDSmemobackenddidit:insteadofcopyingthefirstlinefromthetextintothesummary,itisnowmoved.Inotherwords,thefirstlinegetsstripped.Thechangeisprimarilytechnicallymotivated;bothapproacheshaveprosandcons.\*WebDAV:improvedRadicalesupportRadicale>0.7willreturnstatus200fordeleterequests;isnowtreatedlike204bySyncEvolution.412'PreconditionaFailed'whenaskingtodeleteanalreadyremoveditemistreatedlikethemorecommon404'notfound'.Samewith410'gone'insteadof404whentryingtoreadanon-existentitem.\*filebackend:moreflexiblesyncsupportformemosThedatabaseFormat=text/calendarformemosdidnotsupportsynchronizingasplaintext.WhenusingthenewdatabaseFormat=text/calendar+plain,vCalendar/iCalendar/plaintextareallvalidsyncformats;thestorageisiCalendar2.0VJOURNALinallcases.\*WebDAV:avoidpotentialcrashduringdatabasedetectionWhenaserverrespondstoaPROPFINDforapathwithresultsforsomeotherpath,thenSyncEvolutioncrashedduringthesearchforthedefaultcalendaroraddressbookbecauseofabuginthecodewhichwasmeanttohandlethatkindofresponse.ApparentlyYahooCalendardidthat.NowseenagainincombinationwithRadicale0.6.4.Ingeneral,thecodewasmademorerobusttocopewithbugsinRadicale0.6.4.LaterRadicaleversionsfixedtheseissuesandalsoworkedwithSyncEvolution1.2.2withoutclient-sideworkarounds.\*WebDAV:betterpathnormalization"syncURL"and"database"propertieshadtoendinatrailingslash,otherwiseitemswerenotfound(404errors).Nowthenecessaryslashisaddedautomatically.\*Funambol:avoidslowsyncsinrefreshfromserverlibsynthesishastraditionallyimplemented"refresh-from-server"as"deletelocaldata"plus"slow"sync.Thisismorecompatible,becausesomeservers(likeGoogle)donotsupport"refresh-from-server".Butithasthedownsidethattheservercannotknowthattheclientwon'tsendanydata,andFunambol'sOneMedianowonlyallowsoneslowsyncbeforeblockingthenextoneforacertainperiodoftime.Thisisdonetopreventexcessiveresourceusagebybadlybehavingclients.Toaccomodatebothkindsofservers,thenew"enableRefreshSync"syncpropertycanbesetsettoexplicitlyallowtheusageofthe"refresh-from-server"syncmode.It'soffbydefault.TheFunamboltemplatehasitturnedon,existingconfigsmustbeupdatedmanually(seeupgradingcommentsbelow).*Curltransport:supportSSLServerCertificates=<path>Whenthesettingreferstoadirectory,thenCURLOPT\_CAINFOdoesn'twork(mustbeafile).CheckthisanduseCURLOPT\_CAPATHinstead.Caveat:therearesomecommentsintheAPIdocumentationabout"NSSenabledlibcurl"whichsupportsadirectoryinCURLOPT\_CAINFO.HopefullyprovidinganexplicitpathinCURLOPT\_CAPATHalsoworksinthatconfiguration.\*codecleanup+rewrite:syncingdoneinseparateprocesssyncevo-dbus-servernowrunssyncinginaseparateprocess.Localsyncalsousesasecondhelperprocess.ThismakestheD-BusservermoreresponsiveviaD-Bus(nomoreblockingoperations)andminimizestheeffectofbugsincodeinvolvedwithsyncing(backends,systemlibraries,etc.).Inthelongtermthisrestructuringwillalsoallowmoreadvancedfeatures,likemonitoringlocalorremotestorageforchanges.\*SyncEvolution<->SyncEvolutionsync:multiplecyclespersessionSyncMLonlyallowsonesend/receivecyclepersession.Therearecases(forexample,clientsidemergesdatathatadumberserverfailedtomatchcorrectly)whereclientandserverarestilloutofsyncattheendofacycle.WhenSyncEvolutionsyncswithanotherSyncEvolutioninstance(locallyorremotely),bothsidesdetectthatthepeercancontinuesyncinginthesamesessionandstartoverautomaticallywhenneeded.Previouslytheuserhadtostartanothersyncsessionmanually.Totheuserthisisshownas"numberofcycles"inasyncsessioninthesyncreport."Restart"istheprocessofenteringanewcycle.ThecyclesarealsovisibleinthecommandlineoutputasmultipleINFOlines:
[INFO] eds_contact: starting first time sync from client (peer is server)
[INFO] creating complete data backup of source eds_contact before sync (enabled with dumpData and needed for prin
Local data changes to be applied during synchronization:
*** eds_contact ***
no changes
Maintenance release with various bug fixes. * syncevo-dbus-server + ConnMan: fixed “online” detection ([BMC #21541](https://bugs.meego.com/show_bug.cgi?id=21541), [BMC #24587](https://bugs.meego.com/show_bug.cgi?id=24587)) SyncEvolution did not recognize any cellular connectivity as suitable for syncing. The strict check for certain “connected technology” is unnecessary, anything which makes the computer “online” should be good enough. So now it just uses the ConnMan “State” property. Additional benefit: will continue to work with ConnMan 1.0, which won’t have the “ConnectedTechnologies” property anymore. The Bluetooth available check was also (incorrectly) using the ConnMan API. Now asssume that OBEX/Bluetooth is always available. * automatic backups: added INFO messages and fixed dumpData/printChanges ([BMC #24619](https://bugs.meego.com/show_bug.cgi?id=24619)) Point out that backups are created (user might be unaware otherwise and wonder about the delay), explain why (so that users know how to turn it off). Turning these backups off with dumpData=0 printChanges=0 had to be fixed, backups were always written previously. * EDS compatibility: bumped version check for EDS 3.2 SyncEvolution is known to work with EDS 3.2. Therefore use the libebook/ecal/edataserver libs from 3.2 if available, without warnings in the –version output. Also happens with inconsistent distro setups where the old libs are available and would have been prefered by SyncEvolution 1.2.1 even though the old libs no longer work with EDS 3.2. * GTK-UI: do not accept service config without a username ([BMC #23106](https://bugs.meego.com/show_bug.cgi?id=23106)) Instead of creating such a config, an error dialog is shown. * GTK-UI: updated translations * fixed various compile issues, primarily on Fedora Core 17 (unistd.h/ssize_t, invoking syncevolution during compilation, missing src/dbus/qt/configure-sub.in) SyncEvolution is known to not compile with Bluez 4.97. A patch for Bluez header files is needed to make them work in C++ again, see http://thread.gmane.org/gmane.linux.bluez.kernel/20364 Upgrading from releases before 1.2 ————————————————– Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Release 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Source, Installation, Further information =============================== Source snapshots are in http://downloads.syncevolution.org/syncevolution/sources i386, lpia and amd64 binaries for Debian-based distributions are available via the “stable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:
debhttp://downloads.syncevolution.org/aptstablemain These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/evolution). In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
Maintenance release with various bug fixes. * GTK UI + config: fix “custom server” setup ([BMC #13511](http://https://bugs.meego.com/show_bug.cgi?id=13511)) When the “default” config template (= ScheduleWorld) was downgraded to “not consumer ready” in SyncEvolution 1.1.0.99.1, setting up a custom SyncML service in the GTK UI stopped working because the UI wouldn’t show the “not consumer ready” config. The problem described above is deterministic and fixed now. Initially the problem seemed to be random. So perhaps there is also another, related issue. * phone sync: delete<->delete conflict + phone calendar+todo sync ([BMC #23744](http://https://bugs.meego.com/show_bug.cgi?id=23744)) When deleting an item on phone and locally, the next sync failed with ERROR messages about “object not found”. Retrying the sync then worked. * Nokia: prevent accidental usage of “calendar” or “todo” sources Nokia phones use a combined “calendar+todo” source for syncing. The “calendar” and “todo” sources also exist because that is where local databases are configured. In such a setup, syncing always has to use “calendar+todo”. For example, to refresh from the Linux desktop to the phone, use: –sync refresh-from-server calendar+todo To work with items (restore, show local content), use the underlying sources, as in: –print-items calendar It was possible to accidentally sync with the “calendar”. This commit prevents that by adding an invalid URI setting to the “calendar” and “todo” sources in the Nokia and Ovi templates. Existing configs are not touched, so beware when you already have configured your Nokia phone. * vCard: X- chat extensions were limited to one instance per kind For example, only one Jabber account could be synchronized. This was caused by an incomplete definition of the conversion to and from vCard. * syncevo-dbus-server + phone sync: catch SIGPIPE to avoid premature exit Frederik Elwert reported that running a local sync with a phone via Bluetooth caused the syncevo-dbus-server to shut down during a sync. Explicitly telling the process to ignore the SIGPIPE signal solved that problem. * syncevo-http-server: support chained SSL certificates So far, the file pointed to by –certificate-file had to contain the server certificated (signed by a CA known to the client) and (optionally) a client certificate. Now the file may also contain additional intermediate certificates which will be sent to the client (chained certificates). * documentation: added glossary and command line conventions sections, improved listing of properties, embedd property definitions in man page, README and README.html * EDS compatibility: fixed inconsistency in libecal check The check for the _r variants in libical still used an older max version. This might have prevented using them (if not found) or could have led to a mixture of old and new libecal in the same process (probably crashed). * glib: avoid including glib/*.h headers directly Recent glib deprecates the direct inclusion of some of its headers, in favor of including glib.h. Doing that here whenever possible, so perhaps it now compiles on Fedora 17 (untested). Upgrading from releases before 1.2 ————————————————– Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Release 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Source, Installation, Further information =============================== Source snapshots are in http://downloads.syncevolution.org/syncevolution/sources i386, lpia and amd64 binaries for Debian-based distributions are available via the “stable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:
debhttp://downloads.syncevolution.org/aptunstablemain These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/evolution). In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
With SyncEvolution 1.2 released and work on 1.3 under way it is a good time to take a small break and reflect on the state of the SyncEvolution project. Features ====== This is meant to give an idea of what can be done with SyncEvolution already. For a more complete list of features, see the documentation about [backends](/documentation/backends), [frontends](/documentation/frontends) and [engine](/documentation/syncevolution-and-synthesis-syncml-engine). This section introduces the key ideas, some of the supported protocols + backends and how they work in combination with specific peers. Data Synchronization ——————————- The main purpose of SyncEvolution is the synchronization of Personal Information Management (PIM) data (contacts, events, tasks, notes). This intentionally does not include email and messages in general because those are sufficiently different to require different solutions. SyncEvolution provides *real* synchronization of PIM data: * Items (= contact, event, …) may have different properties and representations on both sides of a sync (heterogeneous environment). * Items can be modified at any time, even while offline (more than just online access to a server). * Also works between devices, without involving a server. * A database (= a set of items) can be synchronized to one or more peers (other devices, servers). For example, the same address book on a desktop can be synchronized with a mobile phone and a laptop. * SyncEvolution supports arbitrary number of databases. However, many peers (in particular SyncML servers) are more limited and only provide access to one addressbook, calendar, etc. For each peer, SyncEvolution needs to remember the previous state so that the next sync can be limited to only those items which have changed. The underlying assumption is that there is no cycle in the connections between multiple sync peers. For example, “phone <-> laptop, laptop <-> server, phone <-> server” is bad because a new item created on the phone will go to the laptop, from there to the server and then come back as new item to the phone, resulting in a duplicate which then repeats the process ad infinitum. Each SyncEvolution instance only knows the peers it is set up to talk with and thus cannot detect the problem. To break this cycle reliably, each item must have a unique ID that is assigned to it once when the item gets created. For ad-hoc synchronization to work without constraints on the topology, this ID must be supported by all involved peers. Unfortunately, calendar and contact data either doesn’t have such an ID or it is not supported. Without such an ID, each request to add a new item must be checked against all existing items based on some key properties to find out whether it is a duplicate. This is slow and error-prone (“Is ‘Doe, John” the same person as ‘John Doe’, or is one the father and the other a child or namesake?”). *TODO* “[ad-hoc synchronization](https://bugs.meego.com/show_bug.cgi?id=23760) between SyncEvolution instances”: * Teach Evolution Data Server to create and preserve a unique ID for contacts (the vCard 3.0 UID gets overwritten at the moment). * Improve the SyncEvolution<->SyncEvolution sync such that * it uses the unique ID to speed up sync between peers which sync against each other for the first time and * checks the ID to find duplicates in following syncs. *TODO* “[ad-hoc synchronization](https://bugs.meego.com/show_bug.cgi?id=23760) between arbitrary peers”: * Do the duplicate detection based on item content for each add request. Testing ———- SyncEvolution has an extensive test suite which is run regularly (used to be nightly, currently triggered manually). It contains unit tests for various aspects of SyncEvolution and does real interoperability testing with different SyncML, CalDAV and CardDAV servers. See [the SyncEvolution 1.2 test report](/blogs/pohly/2011/syncevolution-12-released#comment-525) for an example. More CalDAV/CardDAV servers were added in the 1.3 branch. Keeping this testing going and analyzing/reporting problems is an on-going activity. It is needed to avoid regressions and achieve higher quality in the releases, which are built as part of these test runs. There are additional ideas. *TODO* “improve nightly testing” * Only the sync code is currently checked under valgrind. Also run [syncevo-dbus-server under valgrind](https://bugs.meego.com/show_bug.cgi?id=5609). * Run [tests in MeeGo (or soon Tizen) chroot](https://bugs.meego.com/show_bug.cgi?id=8684). * Automatically [integrate and test](https://bugs.meego.com/show_bug.cgi?id=735) branches which are considered ready for integration. * Test [syncevo-http-server](https://bugs.meego.com/show_bug.cgi?id=1367). Could do [suspend/resume tests](https://bugs.meego.com/show_bug.cgi?id=1009) against that. SyncML + Engine ————————— The Synthesis engine which provides SyncML is one of the best in the industry. It has extensive support for data modeling/conversion and supports suspending a session and resuming it later. The same engine is also used to synchronize between two backends internally. This is how synchronization was added for CalDAV/CardDAV, protocols which themselves only provide online access. This works reasonably well, but there are also quite a few limitations. *TODO* “[libvxx = refactoring of the Synthesis code base](https://bugs.meego.com/show_bug.cgi?id=23761)” * make data conversion available outside of a sync session * decouple sync engine from SyncML and SyncML message encoding/decoding * support items which are a set of items: important for CalDAV + ActiveSync, because those combine all VEVENTs with the same UID in one item; certain transformations cannot be done in the engine at the moment because they depend on access to all related items at once and that is not how they are handled at the moment * more flexible session handling: instead sending changes in one direction, then back and then stopping, allow the session to continue until both sides are in sync; required for CalDAV where storing an event might lead to further changes that have to be sent back *TODO* “[push sync](https://bugs.meego.com/show_bug.cgi?id=23762)”: * react to local or remote changes immediately (instead of polling at long intervals) and/or * make sync sessions without changes more efficient (in particular when polling) *TODO* “[better credential handling](https://bugs.meego.com/show_bug.cgi?id=23763)” * When creating multiple configurations which need the same credentials (Google CalDAV and SyncML, for example), the username/password needs to be set separately. Replace with a mechanism where both configs only contain a pointer to shared credentials. * Also support an external system component which does the authentication without ever returning username/password to SyncEvolution. Depends on having such a component. This could be used to use a CalDAV/CardDAV server as backend for a SyncML server. *TODO* “[use CalDAV/CardDAV inside SyncML server](https://bugs.meego.com/show_bug.cgi?id=23764)”: * In the backend, check databaseuser/password before falling back to the context’s username/password. * Support shared credentials, to avoid having to configure them in each source. At the moment there are situations where SyncEvolution cannot determine what the right resolution for a failed synchronization is. It has a built-in backup mechanism and can ask the user for assistance, but ultimately it would be better to not bother the user. This can be achieved in some cases by simplifying the problem: * The peer (most likely a service on the Internet) must be able to store all data. * There is a separate local database for each database on the peer. In that case it is acceptable to wipe out the local data and restart with the data stored on the peer. *TODO* “[automatic error recovery](https://bugs.meego.com/show_bug.cgi?id=23765)” * Implement the necessary policy in SyncEvolution (“peer wins”). * Check whether this interferes with libfolks (local IDs will change, data added by libfolks to a contact might get lost). CalDAV/CardDAV ————————– Works reasonable well and passes automated testing against a variety of servers (Apple Calendar Server, DAViCal, Google Calendar, Yahoo). But there are some know limitations, like meeting invitations being sent by Evolution and the CalDAV server. *TODO* “[better support for meeting invitations](https://bugs.meego.com/show_bug.cgi?id=23766)” * Find a way to suppress sending of meeting invitations on the CalDAV server when Evolution already sent one, *or* * suppress sending of meeting invitations in Evolution for calendars which are mirrored in a CalDAV server. * When storing a meeting and/or the server does not return an ETag, retrieve the possibly modified item from the server and store the modified item locally. Right now the more possibly automatically modified data on the server gets ignored. Depends on the engine improvements mentioned above. *TODO* “[handle concurrent changes](https://bugs.meego.com/show_bug.cgi?id=23767)” * Use ETags to avoid modifying more recent data on the server. *TODO* “[CalDAV attachments](https://bugs.meego.com/show_bug.cgi?id=23768)” * At the moment, attachments are not supported at all and even may get lost. Need to preserve them and perhaps even support the more efficient “managed attachments” that are currently being discussed by the CalConnect consortium. *TODO* “[WebDAV: use sync extension](https://bugs.meego.com/show_bug.cgi?id=23769)” * Each sync session must list the entire collection (= retrieve path names and ETag) to determine new and modified items. This can be done more efficiently by using the sync extension defined and implemented in the Apple Calendar Server. *TODO* “[CalDAV tasks = VTODO](https://bugs.meego.com/show_bug.cgi?id=24893) * The CalDAV backend is limited to exchanging VEVENTs. It might also not handle collections well (or at all) that contain both VEVENTs and VTODOs. ActiveSync —————- This was not included in SyncEvolution 1.2 although it was already quite usable. Work on it is still going on. The main issue with ActiveSync is the limited data model specified as part of the protocol. Contacts are only allowed to have a fixed number of certain phone numbers and addresses, which is a limitation that neither Evolution nor Google Contacts have. There is no good way to handle these limitations except educating the user about them, or enforcing the same constraints locally by modifying the app which creates and modifies contacts. The ActiveSync calendar format does not support detached recurrences properly (“you are invited to a specific instance of a meeting series”), although Exchange internally does. *TODO* “finish initial ActiveSync support” * [detached recurrences without parent](https://bugs.meego.com/show_bug.cgi?id=22831): * investigate receiving multiple detached recurrences in multiple items with the same UID in each (the Exchange workaround for the ActiveSync calendar format limitation): might break SyncEvolution * implement the “stand-alone detached recurrence” support (either do it like Exchange does or better, create a fake parent event) * test [concurrent item changes](https://bugs.meego.com/show_bug.cgi?id=23770) while a sync runs (supposed to work, but without a test case it is hard to be sure) **[updated]** *TODO* “ActiveSync performance improvement” * Writing changes to the server is done one change at a time. Could be improved considerably by batching changes. Depends on [core engine improvements](https://bugs.meego.com/show_bug.cgi?id=23967). *TODO* “[ActiveSync push sync](https://bugs.meego.com/show_bug.cgi?id=23771)” * enhance activesyncd and SyncEvolution (see engine above) to react to server-side changes with minimal delay *TODO* “[ActiveSync calendar attachments](https://bugs.meego.com/show_bug.cgi?id=23772)” * At the moment, attachments are not supported at all and even may get lost. Need to preserve them. Not sure how to do it efficiently. Google Calendar ———————— Overall Google Calendar syncing works well with CalDAV (I’m using it myself) as long as one does all meeting scheduling in the Google web interface. But there are some known issues, most of them on the server side: * [stand-alone detached recurrences cannot be accessed, SyncEvolution workaround only works for some cases](http://code.google.com/p/google-caldav-issues/issues/detail?id=58) * [meetings removed by SyncEvolution](http://code.google.com/p/google-caldav-issues/issues/detail?id=59) - not sure yet whether this is better fixed on the server or client side * [cannot updated/delete meetings although the organizer](http://code.google.com/p/google-caldav-issues/issues/detail?id=60): SyncEvolution is allowed to create an event, but then cannot update or remove it * [cannot remove detached recurrence](http://code.google.com/p/google-caldav-issues/issues/detail?id=61): breaks the testLinkedItemsRemoveNormal tests Google Contacts ————————- Works via SyncML. Google’s support for SyncML is very incomplete (many properties not supported, for example birthday). *TODO* “better Google Contacts support” * [Test ActiveSync](https://bugs.meego.com/show_bug.cgi?id=23773) and/or * write [backend based on Google Data Protocol](https://bugs.meego.com/show_bug.cgi?id=23774) - might be the best solution, because ActiveSync also has limitations (see above). Apple Calendar Server, DAViCal ———————————————- Done via CalDAV/CardDAV. No know issues with these peers. Yahoo Calendar + Contacts —————————————– Works well, when it works; unfortunately the number of requests per 24 hour period is so limited that the tests cannot complete without running into a 503 “Service Unavailable” error. *TODO* “[Yahoo token authentication](https://bugs.meego.com/show_bug.cgi?id=23775)” * Currently SyncEvolutions uses normal HTTP authentication. Yahoo also supports another, token based authentication mechanism for approved apps. Get SyncEvolution approved and implement that other authentication mechanism. Hopefully that’ll avoid the 503 error. Phones ———– SyncEvolution can synchronize against phones if those phones support SyncML via Bluetooth. This is a common feature among older feature phones but most (all?) Android phones and iPhones don’t support it. Testing of this feature is limited and there is no list of phones which are guaranteed to work. **[updated]** Synchronization with Android/iOS is possible by installing a third-party SyncML client (like the ones from Synthesis) and configuring the phones to use a [SyncEvolution HTTP SyncML server](/wiki/synchronizing-evolution-http-howto). *TODO* “[contact sync via PBAP](https://bugs.meego.com/show_bug.cgi?id=23776)” * Write a backend using PBAP, the only (?) protocol supported by Android and iOS for address book access. Not very good for real syncing, but at least one-way sync should be possible. GTK UI ———– The GTK “sync UI” is the main user interface for SyncEvolution in MeeGo and Linux in general. It is included in the source and binary distribution archives. It supports configuring syncing against SyncML services (extensible via configuration templates) and phones which support SyncML. Configuring the latter is integrated into the GNOME Bluetooth applet, which invokes the sync UI. The UI also has “emergency recovery” support which allows the user to restore from the automatic backup and/or choose between different recovery operations after a failed sync: * continue with local data * continue with remote data * try to merge both (“slow sync”) *TODO* “[port to GTK3](https://bugs.meego.com/show_bug.cgi?id=23094)” * The sync UI is currently using GTK2. It needs to be ported to GTK3. The goal is to put the GTK2 version into maintenance and continue with the GTK3 version. *TODO* “[configure local sync, choose databases](https://bugs.meego.com/show_bug.cgi?id=23777)” * CalDAV/CardDAV and ActiveSync cannot be configured yet with the UI. In this context it becomes important to let the user choose local and remote databases. Would be useful to have, although there is a certain overlap with the Evolution integration. Evolution ————- This happens to be the main backend for storing data locally. Other backends could be supported just as well (there’s nothing technical which favors Evolution) if there were developers motivated enough to implement and test them. KDE/Akonadi has come a long way, but seems less active (see KDE/Akonadi and Community below). *TODO* “Evolution backend improvements” * use new APIs in EDS 3.2/3.4 for more [efficient change detection](https://bugs.meego.com/show_bug.cgi?id=23778) * enable [creating databases](https://bugs.meego.com/show_bug.cgi?id=23779) from inside SyncEvolution again (depends on EDS 3.4 (?)) *TODO* “[integrate SyncEvolution into Evolution](https://bugs.meego.com/show_bug.cgi?id=23780)” * Evolution typically supports offline read access with some of its backends. It does not support write access. SyncEvolution addresses that, but needs to be configured and invoked separately. It would be nice to have seamless and transparent syncing from inside Evolution. ActiveSync is getting integrated like that at the moment -> extend that to CalDAV/CardDAV? File backend —————— There is a generic file backend with a 1:1 mapping between a single item and a file in a directory. The format of the local data can be configured. vCalendar 1.0, iCalendar 2.0, vCard 2.1/3.0 and plain text notes are supported already. *TODO* “[iCalendar 2.0 .ics file](https://bugs.meego.com/show_bug.cgi?id=23781)” * Some people synchronize an .ics file by pointing the Evolution backend to it (no longer works after EDS removed support for the file:// URI) or manipulating the ~/.evolution/calendar data (a hack which only works when being very careful). Write a SyncEvolution backend which reads/writes an iCalendar 2.0 file using libical. KDE/Akonadi ——————– The Akonadi backend is available in the source code. Support for KWallet instead of GNOME keyring is also there. A KDE GUI is in development. *TODO* “[test and release binaries](https://bugs.meego.com/show_bug.cgi?id=23782)” * nightly builds needs to be reconfigured to enable building the KDE support * nightly testing needs to include KDE backends, both for unit testing the backends and real combinations with peers; it depends on the KDE developers to do something if issues are found in those tests (if any are found) Manipulating PIM data ——————————– SyncEvolution also is a very capable tool for manipulating databases via the command line. It can list, import/export and delete items. Converting between formats (for example, mirror Evolution contacts as files in vCard 2.1 format) is possible by setting up synchronization with a file backend as peer. *TODO* “improve item manipulation” * the command line syntax is a bit awkward and depends on dummy configurations: [make config+source names optional](https://bugs.meego.com/show_bug.cgi?id=23783) * [convert between formats](https://bugs.meego.com/show_bug.cgi?id=23784) (depends on libvxx engine improvements) Community ======== Lots of users. The feedback from users is often very helpful for improving the software. Not so many developers, though, except those paid to work on SyncEvolution. Ove Kaaven (port+UI for Maemo/N900 and MeeGo Harmattan/N9/N950) and Frederik Elwert (Genesis UI) are the notable exceptions. Sascha Peilicke/Dinesh/Rohan Garg have done some work for KDE/Akonadi, but are not very active in the SyncEvolution project itself. I can only speculate about the reasons for the lack of external contributors: * The number of people who care about PIM sync and storage is small to start with. It’s not one of those sexy areas that gets a lot of attention, although it is arguably very important. * OpenSync still seems to be considered “the” open source sync solution, despite not having a stable release available or even in sight anytime soon. * Perhaps developers also get the impression that problems will be solved anyway, without having to get involved. That is true for some aspects, but definitely not for all. * SyncEvolution development has a certain learning curve (although Ove and Franz Knipp managed to write their backends with very little assistance) and often happens at a rapid pace, which makes it hard for people to contribute small improvements. * SyncEvolution was focused on Evolution and SyncML initially, which might make it seem too limited in scope for some use cases. But the scope has already increased and will increase further, so now would be a good time to check it out again. Many of the TODOs above will not get worked on unless some external developer picks up the mantle and contributes patches. I’d be more than happy to help someone get started. If there is interest, I can also tag bugs in Bugzilla as “easy fixes”. Right now I am not doing that because it would create additional work which would not be justified if there is no-one interested - I know, chicken and egg…
The major new feature of the 1.2 release is support for non-SyncML protocols in general and CalDAV/CardDAV in particular. ActiveSync support is in development and will be in 1.3. These protocols are implemented as backends which are combined with other backends by SyncEvolution in a so called “local sync”. The GTK sync-ui does not yet support configuring non-SyncML protocols. See the [README.rst and man page](/documentation/syncevolution-usage) for more information on how to use the new feature via the command line. Properties not supported by SyncML servers can now be preserved locally in two-way synchronization ([BMC #15030](https://bugs.meego.com/show_bug.cgi?id=15030)). This depends on information about what properties a SyncML server supports (“CtCap”), which is typically not provided by servers. SyncEvolution contains a copy of that information for Google Contacts ([BMC #15029](https://bugs.meego.com/show_bug.cgi?id=15029)). Akonadi backend and KWallet support were merged. They are not included yet in syncevolution.org binaries. To use them compile from source. The configuration format was updated to solve a conceptual problem inherited with the legacy property names: the “type” property had multiple, sometimes conflicting roles. For example, setting the preferred data format for sync with one peer might have changed the backend selection for some other peer ([BMC #1023](https://bugs.meego.com/show_bug.cgi?id=1023)). Now “backend/databaseFormat/syncFormat/forceSyncFormat” replace “type”. “type” is still accepted by the command line as alias. Upgrading from releases before 1.2 ———————————– Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. Release 1.2 automatically migrates configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Other changes ————- * Using the –sync-property and –source-property command line options is optional, just specifying the property assignment is enough. * syncevo-http-server was enhanced considerably. See http://syncevolution.org/wiki/http-server-howto * support NetworkManager API >= 0.9 ([BMC #19470](https://bugs.meego.com/show_bug.cgi?id=19470)) * syncevolution.org binaries: now compatible with Debian Testing/libnotify.so.4 ([BMC #22668](https://bugs.meego.com/show_bug.cgi?id=22668)) libnotify is not linked directly into syncevo-dbus-server in the syncevolution.org binaries. Instead libnotify.so.1 till .so.4 (current Debian Testing) are opened opened dynamically and the necessary functions are looked up via dlsym(). Not finding the libraries or the functions silently disables this notification mechanism. * Sync mode is recorded when running in SyncML server mode ([BMC #2786](https://bugs.meego.com/show_bug.cgi?id=2786)). * syncevo-dbus-server automatically stops when some of its libraries are updated and restarts if auto-syncing is on ([BMC #14955](https://bugs.meego.com/show_bug.cgi?id=14955)). * Added code for Buteo, mKCal and QtContacts in MeeGo. Buteo and mKCal were removed again from MeeGo, so the code is obsolete. The QtContacts backend may be still be useful to access items via that API, but for syncing on MeeGo the normal EDS backend is used since MeeGo reverted back to EDS as PIM storage. * “databasePassword” source property: lookup failure in keyring ([BMC #22937](https://bugs.meego.com/show_bug.cgi?id=22937)) The databasePassword also wasn’t looked up at all when doing item operations via the command line. When configuring sources for an HTTP server, the config name typically is just the context (@foo). When using the config in the HTTP server, the config name is the peer inside that context (client@foo). Because the GNOME keyring lookup keys for the “databasePassword” (more specifically, the object name) contained the full config name which was different in both cases, looking up the saved password failed. The solution is to normalize the config name (to accomodate for different ways of spelling it) and use only the context, with @ as before. This will break existing setups where the object name in the keyring (incorrectly) includes the full config name. In that case just configure the source again to set the password anew. * Evolution Calendar: fixed detached recurrence support ([BMC #22940](https://bugs.meego.com/show_bug.cgi?id=22940)) When manipulating a meeting series with more than one detached recurrence certain sequences of operations could incorrectly fail with “UID already exists”. * iCalendar 2.0: must set VALUE in EXDATE (part of [BMC #22940](https://bugs.meego.com/show_bug.cgi?id=22940)) EXDATE has a VALUE parameter, which wasn’t defined in the XML profile. Didn’t seem to matter at all in practice, but wasn’t standard-compliant. * GTK sync-ui: wrap sync service descriptions ([BMC #7199](https://bugs.meego.com/show_bug.cgi?id=7199)) Descriptions of different sync services are not fully visible unless word-wrapping gets enabled. * CalDAV/CardDAV + local storage: avoid empty properties The main motivation for this change is that a recent Apple Calendar server rejects vCards with empty BDAY property. Another reason is that keeping the data as small as possible is desirable by itself. Sending an empty property serves as a hint for the peer that the property is supported. This is not necessary when storing an item in a backend. Therefore this commit disables empty properties for all backends which do not themselves set the m_backendRule Synthesis info value. * Google Contacts: ensure that first/middle/name are set when storing in EDS ([BMC #20864](https://bugs.meego.com/show_bug.cgi?id=20864)) Evolution and the MeeGo UX assume that first/middle/last name are set. That is not the case when a contact is created in the Google Contacts web interface. Such contacts are sent by Google without the N property. SyncEvolution now tries to recreate the name components from the FN string, by splitting at word boundaries and assuming “” or “, ” format. Obviously this heuristic fails for some locales. * Evolution Calendar: fixed error handling for broken TZIDs * Sony Ericsson: use ISO-8859-1 for all devices ([BMC #14414](https://bugs.meego.com/show_bug.cgi?id=14414)) Passing invalid UTF-8 strings into libecal caused glib to abort syncevo-dbus-server. * auto sync: show all failed syncs except for temporary network errors ([BMC #21888](https://bugs.meego.com/show_bug.cgi?id=21888)) Notifications were meant to be shown for all errors except temporary ones. This has never been implemented correctly since the feature was introduced: instead of hiding known temporary errors, all errors except 500 (fatal error) were suppressed. * vCard: inline local photo data ([BMC #19661](https://bugs.meego.com/show_bug.cgi?id=19661)) Some platforms (Maemo, MeeGo) store photos in separate files. Now SyncEvolution efficiently includes that photo data in the generated vCard right before sending it to a peer; previously it sent a useless local file:// URI. The Maemo port has a less efficient workaround for that which now should be obsolete. * syncevo-dbus-server: online status wrong without Network Manager or ConnMan ([BMC #21543](https://bugs.meego.com/show_bug.cgi?id=21543)) When neither Network Manager nor ConnMan are running, network presence was “not online”. This prevented running automatic syncs. For developers: * modified backend API - ClientTestConfig modernized - InsertItemResult::m_merged turned from boolean to enum * testing and compilation changes; for example, the minimum version of libsynthesis is now checked at configure time instead of failing at runtime due to missing features in the Synthesis engine SyncEvolution 1.1.99.7 -> 1.2, 13.10.2011 ========================================= Some more bug fixes and testing improvements. * fixed potential invalid memory access in add<->add conflict handling * fixed memory leak in workaround for EDS bug * CalDAV/CardDAV: handle ETags without quotation marks (eGroupware) * updated README: warning about sync direction moved to –sync option Source, Installation, Further information =============================== Source snapshots are in http://downloads.syncevolution.org/syncevolution/sources i386, lpia and amd64 binaries for Debian-based distributions are available via the “stable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:
debhttp://downloads.syncevolution.org/aptstablemain These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in [the download directories](http://downloads.syncevolution.org/syncevolution/evolution). In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
Krzesimir Nowak [wrote about his work](http://krnowak.blogspot.com/2011/10/syncevolution-build-system-work.html) on converting SyncEvolution from an autotools project with recursive make to non-recursive make. Definitely worth a read for anyone interested in autotools. The current SyncEvolution master branch (post 1.2) uses that new build system. His conclusion is that the new system is not necessarily easier to understand than the one before (autotools with some preprocessing shell scripts). Partly that’s because SyncEvolution tries to achieve certain things not supported well by autotools (automatically generated version number, avoid listing all files explicitly, backends which can add to the global configure script), partly it is because non-recursive Automake introduces additional constraints (like having to avoid clobbering variables and rules). Helper scripts are still needed, the only difference is how they get called. Either way, what sold me on the idea of a non-recursive make is that on a machine with many cores, like the SyncEvolution nightly test server, compilation is considerably faster because parallel make can spawn more jobs in parallel. Recursive make often had to wait for the completion of compilation in a sub-directory. If memory serves me right, it was more than twice as fast.
Mostly bug fixes again. Some are a bit more intrusive, thus another pre-release. Changes ======= * syncevolution.org binaries: now compatible with Debian Testing/libnotify.so.4 ([BMC #22668](https://bugs.meego.com/show_bug.cgi?id=22668)) libnotify is not linked directly into syncevo-dbus-server in the syncevolution.org binaries. Instead libnotify.so.1 till .so.4 (current Debian Testing) are opened opened dynamically and the necessary functions are looked up via dlsym(). Not finding the libraries or the functions silently disables this notification mechanism. * calendar sync: better handling for add<->add conflicts (partly fixes [BMC #22783](https://bugs.meego.com/show_bug.cgi?id=22783)) When both sides of a sync have added the same event, the sync must determine which one is more recent instead of blindly overwriting always the same side. Such conflicts are typically rare except for enterprise scenarios where meeting invitiations are processed automatically by a groupware (Exchange, Google Calendar/Mail, …) and then the attendee status is updated on one side. SyncEvolution now does the necessary age comparison and preserves the more recent data for most properties. In some properties the data from both sides is preserved by concatenating the text (description, location, …). It remains to be seen whether that is really desirable. Also, sync statistics are slightly off: the incoming item is counted as “added” even though it gets turned into an update. * item operations: authentication problem for WebDAV when using keyring ([BMC #21311](https://bugs.meego.com/show_bug.cgi?id=21311)) The password still wasn’t looked up in the keyring when using –import/export/delete-items. * “databasePassword” source property: lookup failure in keyring ([BMC #22937](https://bugs.meego.com/show_bug.cgi?id=22937)) The databasePassword also wasn’t looked up at all when doing item operations via the command line. When configuring sources for an HTTP server, the config name typically is just the context (@foo). When using the config in the HTTP server, the config name is the peer inside that context (client@foo). Because the GNOME keyring lookup keys for the “databasePassword” (more specifically, the object name) contained the full config name which was different in both cases, looking up the saved password failed. The solution is to normalize the config name (to accomodate for different ways of spelling it) and use only the context, with @ as before. This will break existing setups where the object name in the keyring (incorrectly) includes the full config name. In that case just configure the source again to set the password anew. * Evolution Calendar: fixed detached recurrence support ([BMC #22940](https://bugs.meego.com/show_bug.cgi?id=22940)) When manipulating a meeting series with more than one detached recurrence certain sequences of operations could incorrectly fail with “UID already exists”. * iCalendar 2.0: must set VALUE in EXDATE (part of [BMC #22940](https://bugs.meego.com/show_bug.cgi?id=22940)) EXDATE has a VALUE parameter, which wasn’t defined in the XML profile. Didn’t seem to matter at all in practice, but wasn’t standard-compliant. * GTK sync-ui: wrap sync service descriptions ([BMC #7199](https://bugs.meego.com/show_bug.cgi?id=7199)) Descriptions of different sync services are not fully visible unless word-wrapping gets enabled. * source configs: don’t check “backend” unless it is needed When using a config which has sources with a backend type set which is not currently available, an error was thrown even if those sources weren’t even part of the current operation (for example, syncing another source which is currently supported). * config migration: avoid name conflicts and auto syncing of old configs ([BMC #22691](https://bugs.meego.com/show_bug.cgi?id=22691)) When (auto-)migrating a config, it was possible that a name for the peer, say foo.old, was chosen for the renamed config although there was already such a config, for example foo.old in ~/.sync4j. Besides being confusing for users, this also led to a bug in the code where it copied from the older config with the foo.old name. The main problem fixed is the disabling of auto syncing in the old config. Otherwise it was still used by syncevo-dbus-server for syncing, which triggered another auto-migration, ad infinitum… * auto syncing: must check whether enabled when looking at unknown URLs (part of [BMC #22691](https://bugs.meego.com/show_bug.cgi?id=22691)) “syncURL = insert your URL here” with “autoSync = 0” did lead to auto sync attempts although it wasn’t enabled. A check for “auto syncing enabled” was missing for the “unknown transport” case. * CalDAV/CardDAV + local storage: avoid empty properties The main motivation for this change is that a recent Apple Calendar server rejects vCards with empty BDAY property. Another reason is that keeping the data as small as possible is desirable by itself. Sending an empty property serves as a hint for the peer that the property is supported. This is not necessary when storing an item in a backend. Therefore this commit disables empty properties for all backends which do not themselves set the m_backendRule Synthesis info value. * Apple CardDAV: apply PHOTO import/export scripts by default A recent Apple Calendar server (correctly) rejects the invalid PHOTO;TYPE=unknown: property in a vCard. This internal representation must be cleared before serializing the field list. * for developers: modified backend API - ClientTestConfig modernized - InsertItemResult::m_merged turned from boolean to enum * testing and compilation changes; for example, the minimum version of libsynthesis is now checked at configure time instead of failing at runtime due to missing features in the Synthesis engine Source, Installation, Further information =============================== Source snapshots are in http://downloads.syncevolution.org/syncevolution/sources i386 and amd64 binaries for Debian-based distributions are available via the “unstable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:
debhttp://downloads.syncevolution.org/aptunstablemain Binaries for lpia will be made available again in the next release. These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in http://downloads.syncevolution.org/syncevolution/evolution. In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
Mostly bug fixes, some improvements in testing and packaging. This release was tested successfully with DAViCal 0.9.9.4. Changes ======= * CalDAV: fixed incorrect change tracking causing “event not found” ([BMC #22329](https://bugs.meego.com/show_bug.cgi?id=22329)) * CalDAV: handle delete<->delete conflict during local sync ([BMC #22327](https://bugs.meego.com/show_bug.cgi?id=22327)) If the same event was deleted both locally and in the CalDAV server, syncing failed with “event not found”. * Google Contacts: ensure that first/middle/name are set when storing in EDS ([BMC #20864](https://bugs.meego.com/show_bug.cgi?id=20864)) Evolution and the MeeGo UX assume that first/middle/last name are set. That is not the case when a contact is created in the Google Contacts web interface. Such contacts are sent by Google without the N property. SyncEvolution now tries to recreate the name components from the FN string, by splitting at word boundaries and assuming “” or “, ” format. Obviously this heuristic fails for some locales. * CalDAV: continue despite Google Calendar access problems (see [BMC #19484](https://bugs.meego.com/show_bug.cgi?id=19484)) An attempt to work around “403 You don’t have access to change that event” errors, perhaps caused by http://code.google.com/p/google-caldav-issues/issues/detail?id=38 The problem is now recorded instead of aborting the sync. The sync then ends in a 22001 = “partial failure” error and the operation will be retried in the next sync. * CalDAV: transform UTC RECURRENCE-ID for Evolution ([BMC #22594](https://bugs.meego.com/show_bug.cgi?id=22594)) Evolution showed a meeting twice on the day of a modified recurrence, if the meeting series was originally created and modified in Exchange, then imported into Google Calendar. * CalDAV syncevolution.org binaries now works when libneon.so.27 or libneon-gnutls.so.27 (Debian) are installed. Previously libneon.so.27 was required, which is no longer available in Debian Testing. * syncevo-dbus-server/gdbus: fixed segfault when asked for properties when none are available ([BMC #22152](https://bugs.meego.com/show_bug.cgi?id=22152)) * Evolution Calendar: fixed error handling for broken TZIDs * Sony Ericsson: use ISO-8859-1 for all devices ([BMC #14414](https://bugs.meego.com/show_bug.cgi?id=14414)) Passing invalid UTF-8 strings into libecal caused glib to abort syncevo-dbus-server. * item operations: authentication problem for WebDAV when using keyring ([BMC #21311](https://bugs.meego.com/show_bug.cgi?id=21311)) The password wasn’t looked up in the keyring when using –print-items/import/export/… * WebDAV: fixed item operations without configuration ([BMC #22164](https://bugs.meego.com/show_bug.cgi?id=22164)) Previously failed with “[ERROR] : virtual read-only configuration node, cannot write property webDAVCredentialsOkay = 1”. * auto sync: show all failed syncs except for temporary network errors ([BMC #21888](https://bugs.meego.com/show_bug.cgi?id=21888)) Notifications were meant to be shown for all errors except temporary ones. This has never been implemented correctly since the feature was introduced: instead of hiding known temporary errors, all errors except 500 (fatal error) were suppressed. * vCard: inline local photo data ([BMC #19661](https://bugs.meego.com/show_bug.cgi?id=19661)) Some platforms (Maemo, MeeGo) store photos in separate files. Now SyncEvolution efficiently includes that photo data in the generated vCard right before sending it to a peer; previously it sent a useless local file:// URI. The Maemo port has a less efficient workaround for that which now should be obsolete. * syncevo-dbus-server: online status wrong without Network Manager or ConnMan ([BMC #21543](https://bugs.meego.com/show_bug.cgi?id=21543)) When neither Network Manager nor ConnMan are running, network presence was “not online”. This prevented running automatic syncs. * fixed compile issues with Debian Testing/gcc 4.6.1 Known issues, might still be resolved for the final 1.2 ====================================== * syncevolution.org binaries: libnotify1 -> libnotify4 incompatibility ([BMC #22668](https://bugs.meego.com/show_bug.cgi?id=22668)) Newer distros no longer have the libnotify.so.1 that syncevolution.org binaries depend on. As a workaround it is possible to install the libnotify1 package from older distro releases. * CalDAV: add<->add conflicts ([BMC #22669](https://bugs.meego.com/show_bug.cgi?id=22669)) Suppose the same meeting invitation for event UID=FOO is processed in both Evolution and Google Calendar. This always happens when the meeting invitation emails is sent to Google Mail, then later viewed in Evolution. On the Evolution side, the invitation is accepted. In Google Calendar this is still open. When syncing in that state the sync engine does not recognize that both sides have added the same meeting and the “meeting accepted” information eventually gets lost. As a workaround, always synchronize the calendar before processing meeting invitation emails. Source, Installation, Further information =============================== Source snapshots are in http://downloads.syncevolution.org/syncevolution/sources i386 and amd64 binaries for Debian-based distributions are available via the “unstable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:
debhttp://downloads.syncevolution.org/aptunstablemain Binaries for lpia will be made available again in the next release. These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in http://downloads.syncevolution.org/syncevolution/evolution. In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
Release 1.1.99.5 is the first release candidate for 1.2. It has gone through a long stabilization period and thus is suitable for normal users. The major new feature of the 1.2 release is support for non-SyncML protocols in general and CalDAV/CardDAV in particular. ActiveSync support is in development. Changes 1.1.1 -> 1.1.99.5 ==================== The major new feature of the 1.2 release is support for non-SyncML protocols in general and CalDAV/CardDAV in particular. ActiveSync support is in development. These protocols are implemented as backends which are combined with other backends by SyncEvolution in a so called “local sync”. The GTK sync-ui does not yet support configuring non-SyncML protocols. See the [README and man page](/documentation/syncevolution-usage) for more information on how to use the new feature via the command line. Properties not supported by SyncML servers can now be preserved locally in two-way synchronization ([BMC #15030](http://bugs.meego.com/show_bug.cgi?id=15030)). This depends on information about what properties a SyncML server supports (“CtCap”), which is typically not provided by servers. SyncEvolution contains a copy of that information for Google Contacts ([BMC #15029](http://bugs.meego.com/show_bug.cgi?id=15029)). Akonadi backend and KWallet support were merged. They are not included yet in syncevolution.org binaries. To use them compile from source. The configuration format was updated to solve a conceptual problem inherited with the legacy property names: the “type” property had multiple, sometimes conflicting roles. For example, setting the preferred data format for sync with one peer might have changed the backend selection for some other peer ([BMC #1023](http://bugs.meego.com/show_bug.cgi?id=1023)). Now “backend/databaseFormat/syncFormat/forceSyncFormat” replace “type”. “type” is still accepted by the command line as alias. Old configurations can still be read. But writing, as it happens during a sync, must migrate the configuration first. In contrast to earlier, more experimental releases in the 1.2 series, 1.1.99.5 and later automatically migrate configurations. The old configurations will still be available (see “syncevolution –print-configs”) but must be renamed manually to use them again under their original names with older SyncEvolution releases. Other changes: * a problem with enabling release mode required replacing 1.1.99.5 with a fixed 1.1.99.5a release * syncevo-http-server was enhanced considerably. See the [HTTP server HOWTO](/wiki/http-server-howto) * support NetworkManager API >= 0.9 ([BMC #19470](http://bugs.meego.com/show_bug.cgi?id=19470)) * Sync mode is recorded when running in SyncML server mode ([BMC #2786](http://bugs.meego.com/show_bug.cgi?id=2786)). * syncevo-dbus-server automatically stops when some of its libraries are updated and restarts if auto-syncing is on ([BMC #14955](http://bugs.meego.com/show_bug.cgi?id=14955)). * Using the –sync-property and –source-property command line options is optional, just specifying the property assignment is enough. * Added support for Buteo, mKCal and QtContacts in MeeGo. Buteo and mKCal were removed again from MeeGo, so the code is obsolete. The QtContacts backend may be still be useful to access items via that API, but for syncing on MeeGo the normal EDS backend is used since MeeGo reverted back to EDS as PIM storage. * code cleanup and various minor fixes/improvements, see ChangeLog Source, Installation, Further information =============================== Source snapshots are in http://downloads.syncevolution.org/syncevolution/sources i386, amd64 and lpia binaries for Debian-based distributions are available via the “unstable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:
debhttp://downloads.syncevolution.org/aptunstablemain These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in http://downloads.syncevolution.org/syncevolution/evolution. In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
David Bremner, the Debian packager of SyncEvolution, [announced](http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=624540) that he is looking for a new maintainer to take over that package. If you care about SyncEvolution in Debian and Ubuntu, then please consider taking over. On this occasion let me thank David Bremner for getting SyncEvolution into Debian and for maintaining it there.
There have been two blog posts recently who point out that data synchronization using open source tools still doesn’t work as well as it should: * [Adam Williamson: “The continuing state of contact + calendar synchronization suck”](http://www.happyassassin.net/2011/04/13/the-continuing-state-of-contact-calendar-synchronization-suck) * [Matěj Cepl: Synchronization sucks!](http://luther.ceplovi.cz/blog/2011/04/synchronization-sucks/) As someone who has worked hard on making synchronization suck less I can’t let this stand without commenting… in particular not when SyncEvolution is mentioned as not being reliable! ;-) Nitpicking ======= First let me point out some factual mistakes in Adam’s post: *”Maemo’s whole synchronization story”* has never been based on SyncEvolution. SyncEvolution is an add-on, supported entirely on a volunteer basis by Ove Kaveh on the device. If people do not get help for SyncEvolution in the Maemo forums, then I can’t do anything about it because I don’t have the time to keep up with everything that is said (or asked) on the web about SyncEvolution. It’s up to the Maemo community to help their peers. SyncEvolution on Linux to N900 [broken somewhere outside of SyncEvolution](https://bugs.meego.com/show_bug.cgi?id=4835) (device, Bluetooth stack) and without support by Nokia for their closed-source sync component on the device I don’t see much chances to fix it, short of some Bluetooth experts getting involved. The statements about MeeGo are also incorrect. Evolution Data Server was not the preferred PIM storage for MeeGo 1.2 until recently, so depending on it for CalDAV/CardDAV support was not an option. Adam points to [a bug report](https://bugs.meego.com/show_bug.cgi?id=319) where I captured some thoughts around the technical aspects. Perhaps this was too brief to be understand without context, but I still think that the arguments and conclusion are valid. More on that below. What exactly is the complaint? ======================= Adam tried SyncEvolution with Horde and eGroupware. Other people were able to get these combinations working, for example just a few days ago [George Runelli with eGroupware](http://www.ruinelli.ch/how-to-sync-egroupware-with-a-tablet-n900-with-syncevolution). It seems to depend a lot on the exact setup on the server side. I had offered Adam to help with diagnosing his problem with Horde (unexpected slow syncs), but he never replied to my email. I’m still convinced that the problem is not in SyncEvolution, but rather on the server side, because SyncEvolution works fine with a variety of other SyncML servers (Funambol, Memotoo, Synthesis, Mobical, to name just those that I test with nightly). It is not SyncEvolution’s fault that the open source groupwares seem to have less stable SyncML support. I tried to work with Horde and eGroupware developers a while back when I started with SyncEvolution. I had a hard time getting anyone to reply to my questions and emails, even when contacting the original developers directly. If the situation is different today, then I’d be happy to restart that effort. I’m not sure what kind of problem Matěj had with SyncEvolution. He doesn’t say in his blog post, only that it does not allow him to reliably sync with his server running Zarafa. I’m not surprised. To the best of my knowledge, the two are unable to synchronize against each other by design, because SyncEvolution is based on SyncML and Zarafa on ActiveSync. So is the complaint that SyncEvolution uses an open protocol and not a patent-encumbered proprietary protocol? Proposed solutions ============== Adam then continues to suggest that the data synchronization model itself is flawed and should be replaced with client/server model where changes are always stored on the server immediately, as in Evolution’s CalDAV and CardDAV backends. This became more clear in an email discussion after he contacted me regarding his blog post. The key difference is this: * True synchronization allows offline modification of the data. * Capable devices by design store a complete copy of the data, without depending on one particular server to remain online. Adam argued that a client/server model can be combined with caching of items and changes. But then the client/server model **becomes** synchronization and must deal with the same kind of problems that it was meant to avoid, like conflicts between items on client and server. Adam [later said](http://luther.ceplovi.cz/blog/2011/04/synchronization-sucks/#comment-37) that changes that cannot be stored anymore should simply be discarded. That doesn’t sound like a very useful approach, because users won’t be able to remember what changes might have been lost and if they do, would most likely be forced to redo them manually. PIM data is more complex then plain text, so the merge strategies that programers know how to use with source code and revision control systems do not apply. Normal users will be even more challenged. The point about not depending on a central server is important, too. In the SyncML world we have recently seen that ScheduleWorld shut down. No data was lost, because by design all users always had a full copy of their data on their own storage. The same can’t be said for all the popular Web 2.0 cloud services… My devices are not always online, for practical and economical reasons. Therefore I want the ability to make changes while offline and will continue to work on the more capable model. SyncML ====== One other aspect is the question whether the data synchronization model itself is flawed, just some protocols implementing it, or only specific implementations of these protocols. I think the approach itself is sound and useful. Adam’s own observation that other implementations of the concept seem to work better confirms that. But SyncML definitely has its flaws, both in the protocol itself and in implementations. SyncML tries to be too flexible for its own good. It allows the implementation of very dumb devices. The downside is an increased complexity on the server side. Because of its open nature, there have been a variety of implementations with varying degrees of capabilities both in the data that is supported and in the quality of the protocol implementation itself. That makes it challenging today to support the whole range of SyncML capable peers. In that sense, SyncML is a victim of its own success. The silver lining ================= I have some hopes today for CalDAV/CardDAV based synchronization. SyncEvolution 1.2 will have support for that, natively. A native implementation has the conceptual advantage that it can use meta data (resource URI + eTag) to speed up change detection, something that wouldn’t be possible when going through Evolution Data Server. CalDAV enforces that each item must have a globally unique ID, which is a considerable simplification for implementing synchronization. CardDAV unfortunately still doesn’t. Good open source implementations exist, for example Apple’s Calendar server. It passes all of the automated SyncEvolution tests. I also hear good things about DAViCal; unfortunately I haven’t found the time to test with it yet. What might be missing in both cases is good integration into a groupware solution, for those who need that.
Quite a bit of spam accumulated in the site comments. Usually I deleted spam comments within a few days after receiving the comment notification email, but not all spam comments triggered those, so I missed quite a lot. I went back and reviewed all comments and deleted as necessary, so the site should be cleaner now. If I missed anything, please let me know. In order to combat spam, all comments are now held for review before being posted. This will lead to delays, so we plan to improve the anti-spam measures and remove the need to review comments manually again. If someone wants to help maintain the site, such help would be highly appreciated! It takes away time that could be spent on improving the software instead…
A Sony Ericsson K800 user reported a problem with non-standard characters (German umlauts in his case). It turned out that the phone uses ISO-8859-1 instead UTF-8 as encoding, without announcing that. It is possible to configure the Synthesis engine so that it does with the character set conversion correctly, but that leads to this question: *which phones need this special treatment*? I’m currently inclined to enable this for all Sony Ericsson phones, unconditionally. If you are using a Sony Ericsson a) which uses something else than ISO-8859-1 as local charset and/or b) which works correctly with non-standard characters already in SyncEvolution 1.1.1, then please leave a comment.
Maintenance release, in particular improving syncing with phones. There was a bug that could cause all kinds of weird behavior after a failed sync with a phone, so updating is highly recommended. Changes 1.1 -> 1.1.1 ==================== * Synthesis engine: fixed a corruption issue in internal meta data which caused duplicates and other problems in a pretty indeterminstic way; apparently caused by failed syncs ([BMC #11044](http://bugs.meego.com/show_bug.cgi?id=11044)). * Synthesis engine: recurrence rules with end date now sent correctly to phones ([BMC #11241](http://bugs.meego.com/show_bug.cgi?id=11241)). The RRULE property was not encoded correctly previously during the iCalendar 2.0 -> vCalendar 1.0 conversion. Events with recurrence count were okay. Probably also affected SyncML servers without iCalendar 2.0 support. The fix was confirmed to work with Nokia phones. It also helps with Sony Ericsson phones, but at least the t700 still has a problem: depending on the phone’s time zone, it repeats the event for one day too long ([BMC #10092](http://bugs.meego.com/show_bug.cgi?id=10092)). * Synthesis engine: fixed broken time zone information when sending to phone; previously that broke sending calendar updates to Nokia phones ([BMC #9600](http://bugs.meego.com/show_bug.cgi?id=9600)). iCalendar 2.0 time zone definitions imported from libical were not encoded correctly in vCalendar 1.0 items as sent to phones. Nokia phones accepted such data when part of a new event, but rejected updates of it. * Synthesis engine: shorter TZIDs, might help N900 calendar ([BMC #6680](http://bugs.meego.com/show_bug.cgi?id=6680)). The shorter TZIDs will be included in iCalendar 2.0 data exported by libsyntesis and thus SyncEvolution. This change is motivated primarily by the observation that the N900 calendar storage can handle TZID=, but not TZID=/softwarestudio.org/Tzfile/. * ScheduleWorld: disable configuration template because service has shut down. The template is only hidden from the GTK sync-ui, but remains in SyncEvolution for the time being because it is referenced in several places. * Evolution CalDAV: added workaround for “must sync twice” ([BMC #10265](http://bugs.meego.com/show_bug.cgi?id=10265)) The Evolution CalDAV backend seems to update its data when closing the database, not when opening it. As a result, syncevolution had to be run twice to see all data changes. The workaround is to open the database twice at the start of the sync. This is done for all calendar databases, regardless of which backend they use, in case that some other (yet unknown) backend needs the same workaround. * GTK sync-ui: workaround for “Sync Now” button not reacting to online status changed ([BMC #9949](http://bugs.meego.com/show_bug.cgi?id=9949)). * Changed slow sync handling. Some users have complained about getting duplicated contacts ([BMC #10081](http://bugs.meego.com/show_bug.cgi?id=10081)). The exact reason is not known (no useful logs provided yet), but it might be due to using “duplicate” as resolution strategy during slow syncs. This caused slightly different contacts to be duplicated instead of merging the two copies, reasoning that “no data loss” is better than “duplicates”. This release switches to a mode where the engine tries harder to avoid duplicates by merging data if modification time stamps are available for contacts (usually they are). When fields differ, the more recent data is kept. * convert absolute alarm back to relative ([BMC #11233](http://bugs.meego.com/show_bug.cgi?id=11233)) Experiments show that at least Nokia phones (and thus perhaps also Mobical.com) interpret a fixed alarm as “repeat alarm with the same relative offset as on first occurrence”. The same transformation to relative alarm times is applied whenever the transformation to absolute alarm is enabled for a peer. * Sony Ericsson: enable conversion to absolute alarm times ([BMC #10092](http://bugs.meego.com/show_bug.cgi?id=10092)) Like Nokia and Mobical.net, Sony Ericsson phones also seem to be unable to deal with relative alarm times - verified with t700. * Sony Ericsson C510: workaround for SyncML violation The phone does not sent identifiers for the target database; using the source identifier as fallback allows a sync to run. * Fixed a regression affecting users who had created a config with SyncEvolution < 1.0. Using the config worked once, then failed with “No configuration for … found”. Users must manually remove the empty “peers” directory inside their affected configuration, the fix only makes configs without that directory usable again ([BMC #9381](http://bugs.meego.com/show_bug.cgi?id=9381)). * Removed obsolete workaround for older mKCal calendar storage. * Fixed error message in QtContacts backend. * Same SYNCEVOLUTION_DEBUG code as in master branch. * Some updates to synccompare, including a workaround for a Perl bug seen on Debian Testing with Perl 5.10.1-16 (Perl panic). * Fix compilation of syncevo-dbus-server with libnotify 0.7.0 ([BMC #10453](http://bugs.meego.com/show_bug.cgi?id=10453)). * Fixed compilation on Debian GNU/Hurd (no MAX_PATH, Mac OS X confusion). Source, Installation, Further information =============================== Source snapshots are in http://downloads.syncevolution.org/syncevolution/sources i386, amd64 and lpia binaries for Debian-based distributions are available via the “stable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:
debhttp://downloads.syncevolution.org/aptstablemain These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in http://downloads.syncevolution.org/syncevolution/evolution. In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
End of November, the scheduleworld.com service shut down. As a user [confirmed later](http://article.gmane.org/gmane.comp.mobile.syncevolution/1868), a notification had been sent earlier to paying customers. So the site is really gone and not just encountering some temporary problems. There does not seem to be any publicly accessible statement about the shutdown, hence this blog post. I have no information about the reasons for discontinuing the service. The notification to users did not give an explanation either. My guess is that the business side of the service did not work out. Syncing is a very challenging, technically difficult problem. I suspect that many users do not understand how much hard work goes into it and thus are less willing to pay for it. Then there is the competition with free services. If users are happy with online access Google Calendar, why should they bother paying for a synchronization service? There are reasons (see below), but perhaps not enough users care. This is really sad, for several reasons. ======================================== First, ScheduleWorld offered a technically superior SyncML implementation for calendar synchronization by supporting the full iCalendar 2.0 semantic, including [UID and RECURRENCE-ID](http://www.estamos.de/blog/2008/06/30/icalendar-20-detached-recurrences/). When describing these properties in 2008, I wrote that not supporting them “looks like a rather significant gap for SyncML-based calendar synchronization compared to proprietary solutions. Hopefully SyncML server vendors will wake up and do something about it…” I don’t think they have - or can anyone point to a SyncML server which supports this today, over one and a half year later? I’m not aware of any. Second, although ScheduleWorld used the open source Funambol server code, none of the extensions or modifications were ever made available. That’s perfectly compliant with the license of the code that was used (GPL instead of AGPL as today), but it implies that the ScheduleWorld source code is now lost and no-one can use or study it. Third, I believe that storing my data on my own hardware and synchronizing it via open standards is crucial. ScheduleWorld made that possible for users who did not want to run their own SyncML server. With true syncing instead of just online access, data is available while offline - pretty obvious, but worth spelling out explicitly. When syncing is based on open standards, users can decide which service they want to use. They don’t depend on that service to store their data; important, because any service might go away unexpectedly and take the data with it. With ScheduleWorld, all data was always and by design also stored locally, so no data was lost when the site shut down. The same is not true for many (if not all) of the now popular Web 2.0 services. Replacements for ScheduleWorld ============================== I’m not sure yet what I should recommend as replacement. Memotoo seems to be popular, but it does not support UID/RECURRENCE-ID semantic, so there are [issues when synchronizing complex iCalendar 2.0 calendar data](http://bugs.meego.com/show_bug.cgi?id=1180). For advanced users the [SyncEvolution server mode](http://syncevolution.org/development/http-server-howto) might be an alternative, but it is in a very rough state at this point and requires a publicly accessible machine for anytime, anywhere synchronization.
debhttp://downloads.syncevolution.org/aptstablemain These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in http://downloads.syncevolution.org/syncevolution/evolution. In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
The current development version of SyncEvolution, the one which will become 1.1, has some new features which may be useful for command line aficionados like myself: query and manipulate items in the databases that can be accessed with SyncEvolution backends. This is particularly useful for the new PIM storages in MeeGo, QtContacts and mkcal (formerly known as KCalExtended). Both are part of MeeGo Core, but come without any kind of frontend in Core. SyncEvolution provides a uniform way of testing these new storages using the command line or scripts. Installation ======== In MeeGo, make sure that at least 1.0.99.5 rpms (required for mkcal) or better, 1.0.99.6 (for QtContacts) get installed. The core “syncevolution” rpm is needed plus “syncevolution-kcalextended” resp. “syncevolution-qtcontacts”. When compiling from source, check out the master branch of [“syncevolution”](http://meego.gitorious.org/meego-middleware/syncevolution) and [“libsynthesis”](http://meego.gitorious.org/meego-middleware/libsynthesis) and follow the instructions in the “HACKING” document. Add –enable-kcalextended resp. –enable-qtcontacts as needed… HOWTO ======= These examples illustrate how to use the new options in combination with mkcal. In 1.0.99.6, “mkcal” can be used as type instead of “kcalextended”, as in the examples below. Only events are supported at the moment, todos and journal entries in the calendar are ignored. Wipe out calendar data the hard way:
rm-f~/.calendardb* Import an event:
$cat>/tmp/example.ics<<EOFBEGIN:VCALENDARPRODID:-//Ximian//NONSGMLEvolutionCalendar//ENVERSION:2.0METHOD:PUBLISHBEGIN:VEVENTDTSTART:20100823T170000DTEND:20100823T173000SUMMARY:firsteventCLASS:PUBLICEND:VEVENTEND:VCALENDAREOF$syncevolution--import/tmp/example.ics--source-propertytype=kcalextended@foobar#0:3f630f1d-8009-4465-907b-3ec28248b90b-rid[ERROR]stderr:kdedate/ksystemtimezone.cpp:313-cannotgetwall_clock_info(localzone)-QDBusError("org.freedesktop.DBus.Error.ServiceUnknown","Thenamecom.nokia.timewasnotprovidedbyany.servicefiles") Because many of these system libraries don’t expect to be used by command line users, they spew out a lot of debugging information. The SyncEvolution command line filters this output and suppresses most of it. Developers who *want* to see it, can set the SYNCEVOLUTION_DEBUG environment variable to 1. They should also add *–daemon=no* to run the operation inside the “syncevolution” process instead of the “syncevo-dbus-server”. The error in this case is [MeeGo bug](http://bugs.meego.com/show_bug.cgi?id=6174 “timed not started”), which I’ll strip from the output of the following examples. The command line above works without any “@foo” configuration and “bar” data source, because the source type is specified explicitly. Configuring this value permanently makes the command line shorter:
$syncevolution--configure--source-propertytype=kcalextended@defaultcalendar List all events:
$syncevolution--daemon=no--print-items@defaultcalendar3f630f1d-8009-4465-907b-3ec28248b90b-rid:firstevent Export events to stdout:
$syncevolution--export-@defaultcalendarBEGIN:VCALENDARPRODID:-//KDesktopEnvironment//NONSGMLlibkcal4.3//ENVERSION:2.0BEGIN:VEVENTCREATED:20100901T131100ZDTSTAMP:20100901T130859ZUID:a62893e3-3df1-4a92-b57b-d05c805d4078LAST-MODIFIED:20100901T131009ZSUMMARY:firsteventDTSTART:20100823T170000DTEND:20100823T173000TRANSP:OPAQUEEND:VEVENTEND:VCALENDAR Export one specific event into a file (like everything else, this also works for multiple events, writing into a single file or a directory):
$syncevolution--export/tmp/exported.ics@defaultcalendar3f630f1d-8009-4465-907b-3ec28248b90b-rid Update that event, showing of reading from stdin here:
`
$ perl -p -e ‘s/first event/first event modified/’ /tmp/example.ics | syncevolution –daemon=no –update - @default calendar 3f630f1d-8009-4465-907b-3ec28248b90b-rid
#0: 3f630f1d-8009-4465-907b-3ec28248b90b-rid
$ syncevolution –daemon=no –export - @default calendar | grep SUMMARY
SUMMARY:first event
Hmm,thatdidn'tworkasintended.Needtoinvestigate...ah,Ihaven'treimplementedthatpartwiththenewmkcalAPIyet.Willbefixedin1.0.99.6.Deletetheevent:
$ syncevolution –delete-items @default calendar 3f630f1d-8009-4465-907b-3ec28248b90b-rid
[INFO] calendar: deleting “first event”
REFERENCE===========
$ syncevolution –help
…
List items:
syncevolution –print-items
Export item(s):
syncevolution [–delimiter ] –export ||- [ …]
Add item(s):
syncevolution [–delimiter |none] –import ||-
Update item(s)
syncevolution –update
syncevolution [–delimiter |none] –update |- …
Remove item(s):
syncevolution –delete-items …
…
–print-items
Shows all existing items using one line per item using
the format “[: ]”. Whether the description
is available depends on the backend and the kind of data that it
stores.
SyncEvolution 1.0.1 ================ A bug fix release. The main reason for releasing it is that SyncEvolution 1.0 no longer worked on recent distros (Fedora Core 13, Debian testing) because of a name clash between the Bluez D-Bus utility code and recent glib. Details: * compile fix for FC 13 (and possibly others): use private copy of gdbus ([BMC #3556](http://bugs.meego.com/show_bug.cgi?id=3556)) * sync-ui: prevent overwriting device configs by accident ([BMC #3566](http://bugs.meego.com/show_bug.cgi?id=3566),[BMC #1194](http://bugs.meego.com/show_bug.cgi?id=1194)) Setting up a phone used the template name as config name and overwrote an existing configuration of another phone that was created using that same template. Now the code uses the Bluetooth device name as set on the device and checks for (less likely) collisions. It also sanitizes the name to avoid complicated config names (only relevant when also using the command line). * syncevo-dbus-server: accept ‘application/vnd.syncml+xml; charset=UTF-8’ for starting an HTTP session ([BMC #3554](http://bugs.meego.com/show_bug.cgi?id=3554)) The redundant charset specification was set by the Funambol Thunderbird client. Because of a literal comparison against ‘application/vnd.syncml+xml’ the messages were rejected. * config fix: operations on non-peer configs failed ([BMC #3157](http://bugs.meego.com/show_bug.cgi?id=3157)) When running operations on a non-peer configuration (like –restore @default addressbook), the operation fails with [ERROR] : type ‘select backend’ not supported * ZYB.com: service goes away end of June 2010, template removed ([BMC #3310](http://bugs.meego.com/show_bug.cgi?id=3310)) * some build ([BMC #2586](http://bugs.meego.com/show_bug.cgi?id=2586), [BMC #3557](http://bugs.meego.com/show_bug.cgi?id=3557)) and language updates Source, Installation, Further information =============================== Source snapshots are in http://downloads.syncevolution.org/syncevolution/sources i386, amd64 and lpia binaries of 1.0 for Debian-based distributions are available via the “stable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:
debhttp://downloads.syncevolution.org/aptstablemain These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in http://downloads.syncevolution.org/syncevolution/evolution. In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
After several betas and lot of testing, it’s finally time to announce the end of the 1.0 development cycle: SyncEvolution 1.0 is released and replaces 0.9.2 as the stable version. 0.1 was released over four years ago. It has always bee part of the long-term vision to bring “personal SyncML” to desktops. Thanks to the Synthesis engine and Intel’s support for the project, this goal has been reached and this release really deserves the magic 1.0 label. For those not familiar with the project, SyncEvolution synchronizes personal information management (PIM) data like contacts, calenders, tasks, and memos using the SyncML information synchronization standard. Up to and including 0.9.2, a third-party SyncML server was required. In 1.0, SyncEvolution itself is able to act as a SyncML server, both via HTTP and Bluetooth (direct sync with phones). SyncEvolution 1.0 ================= Major new features compared to previous stable release: * synchronize directly with a phone over Bluetooth/OBEX * accept Bluetooth/OBEX connections in cooperation with obexd >= 0.19 * run SyncEvolution as a rudimentary HTTP SyncML server The GTK sync-UI can be used to select a paired phone and create a configuration for it based on the bundled configuration templates. Configuration templates are included for Nokia phones; for other phones see the [HOWTO](http://syncevolution.org/development/sync-phone) and check out the Wiki there. Some users have already reported success for Sony Ericsson phones and added setup instructions. New templates from the Wiki can be dropped into ~/.config/syncevolution-templates under an arbitrary file name. Unexpected slow syncs can be detected when running as client ([MB #2416](http://bugzilla.moblin.org/show_bug.cgi?id=2416)) and unless turned off (see “preventSlowSync”), SyncEvolution aborts the session so that the situation can be analyzed. A refresh from client or server might be more suitable. The command line tool provides instructions at the end of its output. The GTK sync-UI points towards its recovery dialog. Automatic synchronization is supported by the syncevo-dbus-server ([MB #6378](http://bugzilla.moblin.org/show_bug.cgi?id=6378)). When that is installed, it will be started as part of a user session and keep running to trigger syncs in the background. Notifications are emitted when syncs start, end or fail ([MB #10000](http://bugzilla.moblin.org/show_bug.cgi?id=10000)). Automatic synchronization can be enabled separately for each peer (“autoSync=0/1”, off by default), will be done at regular intervals (“autoSyncInterval=30” minutes) when online long enough (“autoSyncDelay=5” minutes). That last option ensures that a) an automatic sync does not attempt to use a network connection unless it was already active and b) hopefully is also around long enough to complete the sync. The Synthesis XML configuration was split up into different parts which are assembled from /usr/share/syncevolution/xml. Files in ~/.config/syncevolution-xml override and extend the default files, which my be useful when adding support for a new phone. SyncML servers: * ZYB.com now works thanks to a workaround for anchor handling ([MB #2424](http://bugzilla.moblin.org/show_bug.cgi?id=2424)); only contacts tested because everything else is considered legacy by ZYB.com * Horde: avoid confusing the server with a deviceId that starts like the ones used in old Funambol clients, helps with calendar sync ([MB #9347](http://bugzilla.moblin.org/show_bug.cgi?id=9347)) * Mobical.net (and other, similar services): fix vCalendar 1.0 alarm properties before importing them ([MB #10458](http://bugzilla.moblin.org/show_bug.cgi?id=10458)) * desknow.com works when switching to SyncMLVersion = 1.1 * Funambol, Memotoo (and probably others): preserve meeting series when receiving update for detached recurrence ([MBC #1916](http://bugs.meego.com/show_bug.cgi?id=1916)) Evolution: * calendar backend: minor fix for change tracking when deleting a single instance of a recurring event * workaround for Evolution 2.30: “timezone cannot be retrieved because it doesn’t exist” is triggered incorrectly when importing non-standard timezone definitions because libecal changed an error code ([MB #9820](http://bugzilla.moblin.org/show_bug.cgi?id=9820)) Performance and reliability improvements ([MB #7708](http://bugzilla.moblin.org/show_bug.cgi?id=7708)): * synccompare much faster * database dumps consume less disk space * more intelligent about expiring obsolete session directories and backups * database accesses are reduced in several backends * shorter logs ([MB #8092](http://bugzilla.moblin.org/show_bug.cgi?id=8092)) * message resending helps under unreliable network connectivity (“RetryInterval”) * full support for suspend&resume in SyncEvolution client to SyncEvolution or Synthesis server syncs * better handling of certain third-party time zone definitions ([MBC #1332](http://bugs.meego.com/show_bug.cgi?id=1332)) Improved GTK sync-UI: * revised config screen: all in one list where entries can be expanded, integrated setup of sync with other devices * recovery support: restore from backup, unexpected slow sync handling * spinner while network is in use ([MB #2229](http://bugzilla.moblin.org/show_bug.cgi?id=2229)) * interactive password requests ([MB #6376](http://bugzilla.moblin.org/show_bug.cgi?id=6376)) * uses new D-Bus API Command line: * fixed printing of rejected items ([MB #7755](http://bugzilla.moblin.org/show_bug.cgi?id=7755)) * consistent logging of added/updated/deleted items with short description * improved error reporting (textual descriptions instead of plain error codes [MB #2069](http://bugzilla.moblin.org/show_bug.cgi?id=2069), partial success [MB #7755](http://bugzilla.moblin.org/show_bug.cgi?id=7755), record and show first ERROR encountered [MB #7708](http://bugzilla.moblin.org/show_bug.cgi?id=7708)) * can create new sources ([MB #8424](http://bugzilla.moblin.org/show_bug.cgi?id=8424)) * runs operations inside daemon and thus avoids conflicts with operations done by other clients; for testing purposes (like running a client which talks to a local server in the daemon) it is still possible to ignore the daemon (–daemon=no, [MB #5043](http://bugzilla.moblin.org/show_bug.cgi?id=5043)) * revised README, now also available as man page ([MBC #690](http://bugs.meego.com/show_bug.cgi?id=690)) [Redesigned](http://syncevolution.org/development/direct-synchronization-aka-syncml-server) and reimplemented D-Bus API, used by sync-UI and command line: * central syncevo-dbus-server controls configurations and sync sessions * accepts incoming SyncML connection requests and messages received by independent transport stubs (obexd, HTTP server, …) * can be used by multiple user interfaces at once * fully documented, see src/dbus/interfaces and http://api.syncevolution.org * no longer depends on dbus-glib with hand-written glue code for C++, instead uses gdbus plus automatic C++ binding generated via C++ templates Revised [configuration layout](http://syncevolution.org/development/configuration-handling “design document for configuration handling”) ([MB #8048](http://bugzilla.moblin.org/show_bug.cgi?id=8048)): * several peer-independent sync and source properties are shared between multiple peers * they can be accessed without selecting a specific peer, by using an empty config name or with the new “@” syntax * user interface of command line unchanged * old configurations can be read and written, without causing unwanted slow syncs when moving between stable and unstable SyncEvolution versions * old configurations can be migrated with the “–migrate” command line switch; however, then older SyncEvolution can no longer access them and migrating more than one old configuration causes the second or later configuration to loose its “deviceId” property (which is shared now), causing a slow sync once * config names may contain characters that are not allowed in the file names used for the underlying files; will be replaced with underscores automatically ([MB #8350](http://bugzilla.moblin.org/show_bug.cgi?id=8350)) Upgrading from 0.9.x: * Upgrading and downgrading should work seamlessly when using existing configurations. * The new configuration layout is only used when creating new configurations or explicitly invoking “syncevolution –migrate” (see above). Such configs cannot be used by older SyncEvolution releases. * The new “RetryInterval” property causes messages to be resent after 2 minutes (increased from 1 minute in previous 1.0 betas). At least the Funambol server is known to not handle this correctly in all [cases](http://funzilla.funambol.com/show_bug.cgi?id=7910). So in the Funambol config template the interval is set to zero, disabling the feature. Disabling the feature must be done manually in existing Funambol configurations. SyncEvolution 1.0 beta 3 -> 1.0 final ===================================== Bug fixes and new features: * Configuration templates are stored in a single file ([MBC #1208](http://bugs.meego.com/show_bug.cgi?id=1208)). New templates (like something downloaded from [the Wiki](http://syncevolution.org/wiki) can be dropped into $HOME/.config/syncevolution-templates using an arbitrary file name. * Progress and per-source status are now also reported and recorded when running in server mode ([MBC #1359](http://bugs.meego.com/show_bug.cgi?id=1359)). There are still several limitations (sync mode not reported, no information about sent/received/processed items while the sync runs, see [MBC #2786](http://bugs.meego.com/show_bug.cgi?id=2786)). * Better handling of certain third-party time zone definitions ([MBC #1332](http://bugs.meego.com/show_bug.cgi?id=1332)). Better logging to track down such problems. * D-Bus server + command line: return error code when failed ([MBC #2193](http://bugs.meego.com/show_bug.cgi?id=2193)) * syncevo-phone-config: simplified command line options, several bug fixes (syntax error, incorrect handling of calendar+todo, [MBC #1197](http://bugs.meego.com/show_bug.cgi?id=1197)) * Revised README, now also available as man page ([MBC #690](http://bugs.meego.com/show_bug.cgi?id=690)). Conversion of D-Bus API documentation into .html page ([MBC #1745](http://bugs.meego.com/show_bug.cgi?id=1745)). * Funambol, Memotoo (and probably others): preserve meeting series when receiving update for detached recurrence ([MBC #1916](http://bugs.meego.com/show_bug.cgi?id=1916)) * Fix for potential out-of-bounds memory access ([MBC #1007](http://bugs.meego.com/show_bug.cgi?id=1007)). * HTTP server: fix for potential crash when second session was requested while an older one was still running, initial sync was done without libical time zone information and thus may have mismatched times ([MBC #2435](http://bugs.meego.com/show_bug.cgi?id=2435)) * Nokia E55: convert alarm times ([MBC #1657](http://bugs.meego.com/show_bug.cgi?id=1657)). This is done via a new remote rule in /usr/share/syncevolution/xml/remoterules/server/46_E55.xml If another phone needs the same treatment, then copy that file to ~/.config/syncevolution-xml/remoterules/server and edit the element. * GTK GUI: styling fix ([MBC #1372](http://bugs.meego.com/show_bug.cgi?id=1372)), updated toolbar for MeeGo 1.0 ([MBC #1970](http://bugs.meego.com/show_bug.cgi?id=1970)), avoid duplicating configs when selecting a config created by syncevo-phone-config or the command line ([MBC #1266](http://bugs.meego.com/show_bug.cgi?id=1266)), scroll bars for emergency window ([MBC #1296](http://bugs.meego.com/show_bug.cgi?id=1296)), avoid compile problem on Fedora Core 13 due to name collision with system sync() call, updated translations. Known Issues ============ Compatibility with phones has not been tested as well as compatibility with the officially supported SyncML servers. Some issues have been reported which still need to be investigated: * Nokia N85: ignores refresh-from-server? ([MBC #2722](http://bugs.meego.com/show_bug.cgi?id=2722)) * Sony Ericsson W595: All-day-events created in evolution are synchronized as all-day-events + 1 additional day in the mobile ([MBC #2093](http://bugs.meego.com/show_bug.cgi?id=2093)) * Nokia N81: Evolution contacts loose “Other” email-addresses when synced two-way ([MBC #2566](http://bugs.meego.com/show_bug.cgi?id=2566)) * Nokia phones: absolute alarm time? ([MBC #1657](http://bugs.meego.com/show_bug.cgi?id=1657)) \**Call for action**: [test with your phone](http://syncevolution.org/development/sync-phone “Phone Sync HOWTO”) and [report](http://syncevolution.org/wiki/phone-compatibility-template “Phone Compatibility Wiki page”) which config works and how well synchronization works. Other known issues: * server progress events: no information about sync mode ([MBC #2786](http://bugs.meego.com/show_bug.cgi?id=2786)) * Calendar event alarm synchronization between N900 and Goosync ([MBC #2764](http://bugs.meego.com/show_bug.cgi?id=2764)) Source, Installation, Further information =============================== Source snapshots are in http://downloads.syncevolution.org/syncevolution/sources i386, amd64 and lpia binaries of 1.0 for Debian-based distributions are available via the “stable” syncevolution.org repository. Add the following entry to your /apt/source.list, then install “syncevolution-evolution”:
debhttp://downloads.syncevolution.org/aptstablemain These binaries include the “sync-ui” GTK GUI and were compiled for Ubuntu 8.04 LTS (Hardy). Older distributions like Debian 4.0 (Etch) can no longer be supported with precompiled binaries because of missing libraries, but the source still compiles when not enabling the GUI (the default). The same binaries are also available as .tar.gz and .rpm archives in http://downloads.syncevolution.org/syncevolution/evolution. In contrast to 0.8.x archives, the 1.0 .tar.gz archives have to be unpacked and the content must be moved to /usr, because several files would not be found otherwise. After installation, follow the [getting started](/documentation/getting-started) steps.
SyncEvolution 1.0 beta 3 is available. This release is feature complete and has been tested thoroughly, so only minor bug fixes are expected before releasing 1.0. Beta 3 is ready for day-to-day use and for getting packaged in staging distros as replacement for 0.9.2 or previous betas. If you find issues, please [report them](http://syncevolution.org/support). Binary packages are provided, including Bluetooth support. Major improvements:
automatic, time-driven synchronization in the background
SyncEvolution 0.9.2 and 1.0 beta are available. 0.9.2 replaces 0.9.1 as the official stable release. Binary packages are provided. Changes in both releases are listed below. 1.0 beta contains all changes included in 0.9.2. Development has already shifted towards 1.0, but if you find issues in either version, please report them. In the 1.0 beta, SyncEvolution itself is already able to act as a SyncML server, both via HTTP and Bluetooth (direct sync with phones). Because of incompatible versions of libbluetooth in different distros, the precompiled 1.0 beta packages do not include Bluetooth support. One has to compile from source to use that. **2010-01-30:** if you had problems installing the 1.0 beta binaries because of dependencies on libbluetooth2 or libpcre.so.3, then please try again. Binaries without these dependencies replaced the initial set of binaries today (-3 release instead of -2 for .rpm and .deb). To learn more about SyncEvolution, come to **FOSDEM 2010**:
SyncEvolution 0.9.1 is available. It replaces 0.9 as the official stable release. Changes since that version are listed below. Changes made between beta releases (for those who helped testing them) are documented in the beta release announcement. There are some known issues, see below. The more issues get reported and fixed, the sooner there will be a 0.9.2 maintenance update, so keep the reports coming. If you want to report something or get involved, then contact the team. SyncEvolution synchronizes personal information management (PIM) data like contacts, calenders, tasks, and memos using the SyncML information synchronization standard. Up to and including 0.9.x, a third-party SyncML server is required. In the current “master” branch, SyncEvolution itself is already able to act as a SyncML server. Support for direct synchronization via Bluetooth is planned for 1.0. A snapshot release will be made available as soon as possible - end of October obviously did not work out…
Non-consecutive header level increase; H1 to H3 [myst.header]
SyncEvolution 0.9.1 beta 2 is available, release after fixing all issues which were found in beta 1. It is going to replace 0.9 as the official stable release soon, so this is the chance to find and report problems before they sneak into the final 0.9.1 - die, bugs, die!
Non-consecutive header level increase; H1 to H3 [myst.header]
SyncEvolution 0.9 has been released! The 0.9 version replaces 0.8.1 as the stable version on Linux desktops. Mac OS X and Maemo have not been updated and remain at 0.8.1 (hint: volunteers wanted). 0.9 binaries in .deb, .tar.gz and (for the first time) .rpm format are provided for x86 in 32 and 64 bit mode. Moblin 2.0 comes with SyncEvolution included in the normal Moblin image, with updates provided via the package repositories. Planning and work for SyncEvolution 1.0 is in full swing. 1.0 is intended to add the SyncML server role for direct synchronization with other devices. If you want to get involved, then contact the team: http://syncevolution.org/support
Non-consecutive header level increase; H1 to H3 [myst.header]
This post documents known issues in SyncEvolution 0.9 beta 3 and workarounds. [Update 24.07.2009] The 0.9 beta3 20090723 hotfix solves these issues, so make sure your SyncEvolution is up-to-date.
#4676: error messages caused by incorrect usage of the command line (like typos in the property names) are not printed anymore. Correct command lines work as intended, but both new users (experimenting) and old users (typos) are likely to make mistakes occasionally. No workaround, needs to be fixed as soon as possible. Run with SYNCEVOLUTION_DEBUG=1 in your environment to see the error messages again. #4551: SSL certificate check with libsoup/gnutls currently fails for Google. The last-minute workaround for the SyncEvolution Google configuration template was to use http instead of https. It turned out that Google does not provide SyncML without SSL. That means that the default configuration has to be modified. This cannot be done via the GUI. Set up the configuration either via GUI or command line. Then use the command line to update the configuration:
syncevolution--configure\--sync-propertysyncURL=https://m.google.com/syncml\--sync-propertySSLVerifyHost=0\--sync-propertySSLVerifyServer=0\google
The end is near - SyncEvolution 0.9 is almost done. For the first time in the 0.9 series, precompiled binaries are made available again together with the new 0.9 beta 3 source snapshot. Users are encouraged to upgrade now and give feedback before the final 0.9 release. See below for upgrade and install instructions. The initial 0.9 beta 3 packages had
two problems which were discovered while testing them. Make sure that you install the updated 0.9 beta 3 20090723 version.
SyncEvolution 0.9 beta 3 is released and I still haven’t found the time to try out whether the 0.9 series still compiles on Maemo and Mac OS X. Unless someone with some skills in compiling software on those platforms and the necessary motivation steps up, these platforms might not be supported in the initial 0.9 release. The HTTP transport from 0.8, libcurl, is still supported and compiled regularly. Internal API changes were made in the Mac OS X backend as part of the normal maintenance, without testing them yet. So chances are that it will compile and run without too much effort. If you are interested, the HACKING document contains sections on how previous releases were compiled - give it a try and help out fellow users by sharing the result!
Update 02.11.2009: a mail asking for help was sent to the Maemo developers list. The Mac OS X port requires a bit more changes, but there has been some progress. A permanent maintainer would still be needed, though.
The SyncEvolution team is pleased to announce our new web site and place to interact with the SyncEvolution community. Most of the content and documentation from our previous site (http://estamos.de) has been migrated here. Let us know what you think!
Non-consecutive header level increase; H1 to H3 [myst.header]