Wednesday , 26 September 2018
Latest topics!!

How to do IMAP Sync, Email migration using imapsync

Email migration is always a head ache when it comes to do between platforms like windows to linux.  Also we may face issues, when it is done using migration tools sometimes mail will not get migrated completely.

We can do the email migration easly from whatever platforms to linux using IMAPSync. Since it is done based on protocol(IMAP) and not depending on platforms it is easy and most reliable way.

Here I will explain the things with a scenario.

You want to migrate an email account with thousands of mails from plesk sever to a cpanel server. Here source server is plesk server and the destination server is cpanel. First thing you have to do here is install “imapsync” in the destination server(cpanel server). Follow the below steps for the same:

Centos 7

First install access to the Epel repository via yum:

yum install epel-release

Then install imapsync and its dependencies:

yum install imapsync

After installing imapsync, it should be able to work on your system.
A good test that shows also the basic example:

imapsync

A live test:

imapsync --testslive

Unit tests:

imapsync --tests

Centos 6

First, install access to the Epel repository

wget http://dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
rpm -Uvh epel-release-6-8.noarch.rpm

Then install imapsync and its dependencies:

yum install imapsync

After installing imapsync, it should be able to work on your system.
A good test that shows also the basic example:

imapsync

A live test:

imapsync --testslive

Unit tests:

imapsync --tests

Once the installation is competed whitelist IP address of plesk server in the cpanel server and vice-versa.

Second is get the hostname, email address and password of the email account you want to migrate from the plesk server. Let it be as follows:

hostname : host1.plesk.com
email account : email@plesk.com
password : my4tr00nGp@sswoord

Third is create the email account in the cpanel server. Let it be as follows:

hostname : host2.cpanel.com
email account : email@cpanel.com
password : my4tr00nGp@sswoord

Go to home directory of the server

cd /home

create the file imap_sync.sh

Insert the content as follows:

imapsync --host1 host1.plesk.com --user1 email@plesk.com --password1 'my4tr00nGp@sswoord' \
--host2 host2.cpanel.com --user2 email@cpanel.com --password2 'my4tr00nGp@sswoord' \

Make it executable

chmod +x imap_sync.sh

Then run it as

bash /home/imap_sync.sh

The Sample output should be as follows:

[root@aries home]# bash imap_sync.sh
Transfer started at Sun Mar 19 12:49:03 2017
PID is 30331
Log file is LOG_imapsync/2017_03_19_12_49_03_hari@twinbeeservers.com.txt ( to ch
ange it, use --logfile path ; or use --nolog to turn off logging )
$RCSfile: imapsync,v $ $Revision: 1.684 $ $Date: 2016/03/17 08:35:03 $
Here is a [linux] system (Linux aries.twinbeeservers.com 2.6.32-042stab120.18 #1
SMP Fri Jan 13 10:32:04 MSK 2017 x86_64)
With perl 5.10.1 Mail::IMAPClient 3.37
Command line used:
/usr/bin/imapsync --host1 176.9.193.61 --user1 hari@hari.com --password1 MASKED
--host2 176.9.193.56 --user2 hari@twinbeeservers.com --password2 MASKED
Temp directory is /tmp ( to change it use --tmpdir dirpath )
PID file is /tmp/imapsync.pid ( to change it use --pidfile filepath ; to avoid i
t use --pidfile "" )
/tmp/imapsync.pid already exists, overwriting it ( use --pidfilelocking to avoid
concurrent runs )
Modules version list:
Mail::IMAPClient 3.37
IO::Socket 1.31
IO::Socket::IP ?
IO::Socket::INET 1.31
IO::Socket::SSL 1.31
Net::SSLeay 1.35
Compress::Zlib 2.021
Digest::MD5 2.55
Digest::HMAC_MD5 1.01
Digest::HMAC_SHA1 1.01
Term::ReadKey 2.30
File::Spec 3.3
Time::HiRes 1.9721
Unicode::String 2.09
IO::Tee 0.64
File::Copy::Recursive 0.38
Authen::NTLM 1.09
URI::Escape 3.31
Data::Uniqid 0.12
JSON ?
JSON::WebToken ?
Crypt::OpenSSL::RSA ?
LWP 6.15
HTML::Entities 3.64
Filesys::DfPortable ?
( use --no-modules_version to turn off printing this Perl modules list )
Info: turned ON syncinternaldates, will set the internal dates (arrival dates) o
n host2 same as host1.
Host1: will try to use LOGIN authentication on host1
Host2: will try to use LOGIN authentication on host2
Host1: imap connexion timeout is 120 seconds
Host2: imap connexion timeout is 120 seconds
Host1: IMAP server [176.9.193.61] port [143] user [hari@hari.com]
Host2: IMAP server [176.9.193.56] port [143] user [hari@twinbeeservers.com]
Host1: connecting and login on host1 [176.9.193.61] port [143] with user [hari@h
ari.com]
Host1 banner: * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENA
BLE IDLE NAMESPACE STARTTLS AUTH=PLAIN AUTH=LOGIN] Dovecot ready.
Host1: 176.9.193.61 says it has CAPABILITY for AUTHENTICATE LOGIN
Host1: success login on [176.9.193.61] with user [hari@hari.com] auth [LOGIN]
Host2: connecting and login on host2 [176.9.193.56] port [143] with user [hari@t
winbeeservers.com]
Host2 banner: * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENA
BLE IDLE NAMESPACE STARTTLS AUTH=PLAIN AUTH=LOGIN] Dovecot ready.
Host2: 176.9.193.56 says it has CAPABILITY for AUTHENTICATE LOGIN
Host2: success login on [176.9.193.56] with user [hari@twinbeeservers.com] auth
[LOGIN]
Host1: state Authenticated
Host2: state Authenticated
Host1 capability: IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT
SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND UR
L-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1
CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BIN
ARY MOVE NAMESPACE NOTIFY SPECIAL-USE COMPRESS=DEFLATE QUOTA
Host2 capability: IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT
SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND UR
L-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1
CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BIN
ARY MOVE NAMESPACE NOTIFY SPECIAL-USE COMPRESS=DEFLATE QUOTA

Host2: found quota, presented in raw IMAP
Sending: 4 GETQUOTAROOT INBOX
Sent 22 bytes
Read: * QUOTAROOT INBOX Mailbox
* QUOTA Mailbox (MESSAGE 7 2147483647)
4 OK Getquotaroot completed (0.000 + 0.000 secs).

Host2: Quota current storage is 0 bytes. Limit is 0 bytes. So 0 % full
Host1: found 7 folders.
Host2: found 7 folders.
Including all folders found by default. Use --subscribed or --folder or --folder
rec or --include to select specific folders. Use --exclude to unselect specific
folders.
Host1: checking all wanted folders exist.
Host1: checking all wanted folders are selectable. Use --nocheckselectable to av
oid this check.
Host1: guessing separator from folder listing: [.]
Host1: separator given by NAMESPACE: [.]
Host2: guessing separator from folder listing: [.]
Host2: separator given by NAMESPACE: [.]
Host1: guessing prefix from folder listing: [INBOX.]
Host1: prefix given by NAMESPACE: [INBOX.]
Host2: guessing prefix from folder listing: [INBOX.]
Host2: prefix given by NAMESPACE: [INBOX.]
Host1 separator and prefix: [.][INBOX.]
Host2 separator and prefix: [.][INBOX.]
Turned off automapping folders ( use --automap to turn on automapping )

++++ Listing folders
All foldernames are presented between brackets like [X] where X is the foldernam
e.
When a foldername contains non-ASCII characters it is presented in the form
[X] = [Y] where
X is the imap foldername you have to use in command line options and
Y is the uft8 output just printed for convenience, to recognize it.

Host1 folders list:
[INBOX]
[INBOX.Archive]
[INBOX.Drafts]
[INBOX.Junk]
[INBOX.Sent]
[INBOX.Trash]
[INBOX.spam]

Host2 folders list:
[INBOX]
[INBOX.Archive]
[INBOX.Drafts]
[INBOX.Junk]
[INBOX.Sent]
[INBOX.Trash]
[INBOX.spam]

Folders mapping from --automap feature (use --f1f2 to override any mapping):

Folders mapping from --f1f2 options, it overrides --automap:
Folders sizes before the synchronization.
You can remove foldersizes listings by using "--nofoldersizes" and "--nofolders
izesatend"
but then you will also loose the ETA (Estimation Time of Arrival) given after ea
ch message copy.
++++ Calculating sizes of 7 folders on Host1
Host1 folder 1/7 [INBOX] Size: 56313 Message
s: 3 Biggest: 54141
Host1 folder 2/7 [INBOX.Archive] Size: 0 Message
s: 0 Biggest: 0
Host1 folder 3/7 [INBOX.Drafts] Size: 0 Message
s: 0 Biggest: 0
Host1 folder 4/7 [INBOX.Junk] Size: 0 Message
s: 0 Biggest: 0
Host1 folder 5/7 [INBOX.Sent] Size: 1478 Message
s: 3 Biggest: 498
Host1 folder 6/7 [INBOX.Trash] Size: 0 Message
s: 0 Biggest: 0
Host1 folder 7/7 [INBOX.spam] Size: 0 Message
s: 0 Biggest: 0
Host1 Nb folders: 7 folders
Host1 Nb messages: 6 messages
Host1 Total size: 57791 bytes (56.437 KiB)
Host1 Biggest message: 54141 bytes (52.872 KiB)
Host1 Time spent: 0.9 seconds
++++ Calculating sizes of 7 folders on Host2
Host2 folder 1/7 [INBOX] Size: 114164 Message
s: 4 Biggest: 57851
Host2 folder 2/7 [INBOX.Archive] Size: 0 Message
s: 0 Biggest: 0
Host2 folder 3/7 [INBOX.Drafts] Size: 0 Message
s: 0 Biggest: 0
Host2 folder 4/7 [INBOX.Junk] Size: 0 Message
s: 0 Biggest: 0
Host2 folder 5/7 [INBOX.Sent] Size: 1478 Message
s: 3 Biggest: 498
Host2 folder 6/7 [INBOX.Trash] Size: 0 Message
s: 0 Biggest: 0
Host2 folder 7/7 [INBOX.spam] Size: 0 Message
s: 0 Biggest: 0
Host2 Nb folders: 7 folders
Host2 Nb messages: 7 messages
Host2 Total size: 115642 bytes (112.932 KiB)
Host2 Biggest message: 57851 bytes (56.495 KiB)
Host2 Time spent: 0.0 seconds
++++ Looping on each one of 7 folders to sync
1/7 [INBOX] -> [INBOX]

2/7 [INBOX.Archive] -> [INBOX.Archive]

3/7 [INBOX.Drafts] -> [INBOX.Drafts]

4/7 [INBOX.Junk] -> [INBOX.Junk]

5/7 [INBOX.Sent] -> [INBOX.Sent]

6/7 [INBOX.Trash] -> [INBOX.Trash]

7/7 [INBOX.spam] -> [INBOX.spam]

++++ End looping on each folder

Folders sizes after the synchronization.
You can remove this foldersizes listing by using "--nofoldersizesatend"
++++ Calculating sizes of 7 folders on Host1
Host1 folder 1/7 [INBOX] Size: 56313 Message
s: 3 Biggest: 54141
Host1 folder 2/7 [INBOX.Archive] Size: 0 Message
s: 0 Biggest: 0
Host1 folder 3/7 [INBOX.Drafts] Size: 0 Message
s: 0 Biggest: 0
Host1 folder 4/7 [INBOX.Junk] Size: 0 Message
s: 0 Biggest: 0
Host1 folder 5/7 [INBOX.Sent] Size: 1478 Message
s: 3 Biggest: 498
Host1 folder 6/7 [INBOX.Trash] Size: 0 Message
s: 0 Biggest: 0
Host1 folder 7/7 [INBOX.spam] Size: 0 Message
s: 0 Biggest: 0
Host1 Nb folders: 7 folders
Host1 Nb messages: 6 messages
Host1 Total size: 57791 bytes (56.437 KiB)
Host1 Biggest message: 54141 bytes (52.872 KiB)
Host1 Time spent: 0.0 seconds
++++ Calculating sizes of 7 folders on Host2
Host2 folder 1/7 [INBOX] Size: 114164 Message
s: 4 Biggest: 57851
Host2 folder 2/7 [INBOX.Archive] Size: 0 Message
s: 0 Biggest: 0
Host2 folder 3/7 [INBOX.Drafts] Size: 0 Message
s: 0 Biggest: 0
Host2 folder 4/7 [INBOX.Junk] Size: 0 Message
s: 0 Biggest: 0
Host2 folder 5/7 [INBOX.Sent] Size: 1478 Message
s: 3 Biggest: 498
Host2 folder 6/7 [INBOX.Trash] Size: 0 Message
s: 0 Biggest: 0
Host2 folder 7/7 [INBOX.spam] Size: 0 Message
s: 0 Biggest: 0
Host2 Nb folders: 7 folders
Host2 Nb messages: 7 messages
Host2 Total size: 115642 bytes (112.932 KiB)
Host2 Biggest message: 57851 bytes (56.495 KiB)
Host2 Time spent: 0.0 seconds
++++ Statistics
Transfer started on : Sun Mar 19 12:49:03 2017
Transfer ended on : Sun Mar 19 12:49:05 2017
Transfer time : 3.0 sec
Folders synced : 7/7 synced
Messages transferred : 0
Messages skipped : 6
Messages found duplicate on host1 : 0
Messages found duplicate on host2 : 0
Messages void (noheader) on host1 : 0
Messages void (noheader) on host2 : 0
Messages deleted on host1 : 0
Messages deleted on host2 : 0
Total bytes transferred : 0 (0.000 KiB)
Total bytes duplicate host1 : 0 (0.000 KiB)
Total bytes duplicate host2 : 0 (0.000 KiB)
Total bytes skipped : 57791 (56.437 KiB)
Total bytes error : 0 (0.000 KiB)
Message rate : 0.0 messages/s
Average bandwidth rate : 0.0 KiB/s
Reconnections to host1 : 0
Reconnections to host2 : 0
Memory consumption : 250.4 MiB
Biggest message : 0 bytes
Start difference host2 - host1 : 1 messages, 57851 bytes (56.495 KiB)
Final difference host2 - host1 : 1 messages, 57851 bytes (56.495 KiB)
Detected 0 errors
Homepage: http://imapsync.lamiral.info/
Log file is LOG_imapsync/2017_03_19_12_49_03_hari@twinbeeservers.com.txt ( to ch
ange it, use --logfile filepath ; or use --nolog to turn off logging )

Some times you will run into issues and will see the following error:
Host1: connecting and login on host1 [176.9.193.61] port [143] with user [hari@hari.com] Host1 failure: can not open imap connection on host1 [176.9.193.61] with user

This is because you have missed the most important step, Did not whitlist IPs in both servers. Go ahead and do the same
Done!!

Have questions? Click here to submit a support request.

About Harikrishnan M

Working as Senior System Administrator at Twinbeeservers.

Leave a Reply

Your email address will not be published. Required fields are marked *

*