1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Fixed Duplicate emails on import

Discussion in 'Resolved Bug Reports' started by emma57573, Apr 24, 2012.

  1. emma57573

    emma57573 Member

    I have a bit of an issue
    Im importing from PHPBB3 however my user table is separate from my PHPBB table as I use the user details for other aspects of my site so its custom but originates from phpbb so uses the same basis but is has a different name and some of the columns are different.

    I have changed the importer code to reflect this and ran a test import on a fresh install and it worked but with one issue.

    XF importer requires us to merge or change duplicate emails, I merged for the sake of the test import. HOWEVER I have quite a few users with the same email address. They used to be allowed couple of accounts with the same email this is because users open shops on my site and some users require more than one shop as they have a couple of business that they like to keep separate and I only allow them to have one shop per account due to the way the site is set up.

    I no longer allow this users that want more than one shop are required to select a new email address. However that doesn't change the fact that I have duplicate emails that I don't want to merge or change.

    Is there a way around this? I don't really want to email them all asking for alternative addresses as there is just over 100 users. I would rather just force the import to import them and then bulk email them after to ask them to change it.

    If I change the import script to force it to ignore the duplicates will I cause problems elsewhere?
     
  2. Jake Bunce

    Jake Bunce XenForo Moderator Staff Member

    XenForo requires unique emails because it lets users login using their email address. So you can't just force the system to accept duplicate emails.

    If you don't want to use the auto-merge option then I suggest manually dealing with the duplicates in your phpBB database before importing.
     
  3. emma57573

    emma57573 Member

    arrrh thats so not what I wanted to hear! This should be fun!!
     
  4. emma57573

    emma57573 Member

    Spent ages sorting this out checking usernames and emails.
    Checked so that 100% their would be no conflicts only to import last night and the import found conflicting usernames!!

    Really confused, so I checked both databases before choosing options there was only one of the usernames in the database and it had not yet been imported.
    So what I had to do to import them was change the username to MERGE_username there was about 50 of them.

    I checked xf_users after for all the names starting with MERGE_ and for each of those I checked for duplicate names and found none! so I took the merge part out and corrected the username for all of them. Then again I checked for duplicated and checked to make sure the user_id matched as expected.

    What I think has happened is that the importer is checking the user name where 'username1' and 'username11' would match and asks you to merge, when infact they are completely different users. This could cause problems with users being merged with completely different users if someone merges without checking the database. I think this should be looked at as a possible bug?
     
  5. emma57573

    emma57573 Member

    Just checked again to check the names of the ones I had an issue with and every one I had a problem with had a '1' after the username like 'username1' defiantly none of them where duplicates and they all had this common theme so must be a bug.
     
  6. Jake Bunce

    Jake Bunce XenForo Moderator Staff Member

    Did you import into a new installation of XenForo? If the usernames already existed from a previous import then it is expected that it finds duplicates for all accounts when you import them again. Incremental imports are not supported. Each time you run the import it will import everything, not just new stuff. That's why it is best to import into a new installation of XenForo each time.
     
  7. emma57573

    emma57573 Member

    No completely clean install and as I said I checked before doing anything with the suggested merge and they did not exist in xf_users and there was only one of each in the FORUM_users table but each one was a user ending in the number 1
    Its a common theme too much to be a coincidence as there was over 50
     
  8. Jake Bunce

    Jake Bunce XenForo Moderator Staff Member

    The username check goes by exact match, not a partial match. The two possible conflicts are:

    1) Duplicate usernames.

    2) Duplicate emails.

    The duplicates can be between a phpbb user and a XF user, or between two phpbb users as they are being imported into XF.

    I have never encountered any bugs in the importer that exceed these two possibilities. If you believe there is a bug then I can test your import if you give me a copy of your database.
     
  9. emma57573

    emma57573 Member

    Its odd but they defiantly were not duplicates and were coming from one import of FORUM_users. I can't really give you my database or table as theres loads of personal info etc that I can't give out as my sites not just a forum. However Im off out at the moment but when I get back I can provide you with enough details for you to be able to test.

    I have over 17,500 users and it only effected 50 users and every single one of them ended with the number 1. No duplicate emails as expected just usernames that end with 1. As I say I changed them all to MERGE_someusername1 to get round it then went through the database checking for MERGE_ took the merge part out and then checked to see if the username was duplicated and it was not. Very time consuming but thats how I know they were not duplicated.
     
  10. Jake Bunce

    Jake Bunce XenForo Moderator Staff Member

    [​IMG]

    library/XenForo/Importer/PhpBb3.php

    The red code is of interest. The blue piece will cause it to remove 1's from the end of the username when performing the dupe check, which matches perfectly with your description of the problem. This must be a mistake in the code as I can't think of any good reason for this.

    Try removing the blue piece. Then run your import again. I suspect that will fix your problem. It if does fix the problem then this should be moved to the bugs forum.

    Code:
    	protected function _importOrMergeUser(array $user, array $options = array())
    	{
    		$im = $this->_importModel;
    
    		if ($user['user_email'] && $emailMatch = $im->getUserIdByEmail($this->_convertToUtf8($user['user_email'], true)))
    		{
    			if (!empty($options['mergeEmail']))
    			{
    				return $this->_mergeUser($user, $emailMatch);
    			}
    			else
    			{
    				if ($im->getUserIdByUserName($this->_convertToUtf8($user['username'], true)))
    				{
    					$this->_session->setExtraData('userMerge', $user['user_id'], 'both');
    				}
    				else
    				{
    					$this->_session->setExtraData('userMerge', $user['user_id'], 'email');
    				}
    				return false;
    			}
    		}
    
    		$name = utf8_substr($this->_convertToUtf8(rtrim($user['username'], true)), 0, 50);
    
    		if ($nameMatch = $im->getUserIdByUserName($name))
    		{
    			if (!empty($options['mergeName']))
    			{
    				return $this->_mergeUser($user, $nameMatch);
    			}
    			else
    			{
    				$this->_session->setExtraData('userMerge', $user['user_id'], 'name');
    				return false;
    			}
    		}
    
    		return $this->_importUser($user, $options);
    	}
    
     
  11. Jake Bunce

    Jake Bunce XenForo Moderator Staff Member

    For the devs... the same code exists in the vBulletin importer.
     
  12. emma57573

    emma57573 Member

    I didn't want to do a whole new import as I fixed it as I went along, But I did just test that bit of the script and got these results:

    Code:
        $user= 'username1';
            $name = utf8_substr(rtrim($user, true), 0, 50);
           
    echo 'input_name : '; echo $user;  /// Result = 'username1'
     
    echo 'output_name : '; echo $name; /// Result = 'username'
    So yes looks like thats the nail on the head. I don't think I can move to bugs so if someone could that would be great.
     
  13. Mike

    Mike XenForo Developer Staff Member

    The true is supposed to be on the convertToUtf8 call, so fixed in the code now.
     

Share This Page