Console Import

Vincent Gabriel

Active member
Hi,

Looking for some feedback regarding console command import. i've been working with it and so far it works pretty well, anyone has any benchmarks or timing they've done using the importer tool?

For some reason if i uncomment the mysql transactions from the steps it appears to work faster and process the results quicker, maybe just my testing.

I only tested with VBulletin and i have a large database to test with 20M posts 300K users.

I'd also like to start a discussion on command tools for large forums that have ssh access and running on vps/dedicated servers and much rather use command tools than web based.


Thoughts?

Thank You.
 

Attachments

  • Screen Shot 2016-02-01 at 3.02.43 PM.png
    Screen Shot 2016-02-01 at 3.02.43 PM.png
    313.9 KB · Views: 6
  • Screen Shot 2016-02-01 at 5.12.36 PM.png
    Screen Shot 2016-02-01 at 5.12.36 PM.png
    119.9 KB · Views: 6
  • Screen Shot 2016-02-01 at 5.13.25 PM.png
    Screen Shot 2016-02-01 at 5.13.25 PM.png
    151.5 KB · Views: 5
  • Screen Shot 2016-02-01 at 5.15.32 PM.png
    Screen Shot 2016-02-01 at 5.15.32 PM.png
    432.1 KB · Views: 5
  • Screen Shot 2016-02-01 at 12.36.45 PM.png
    Screen Shot 2016-02-01 at 12.36.45 PM.png
    313.1 KB · Views: 5

Vincent Gabriel

Active member
Did anyone ever try to benchmark or profile the importer? i am still working on trying to make this work for large imports but the only thing that takes forever right now is the threads, i'll try and profile to see what's the most time consuming operation for that but i kinda think it'll be the lookups for the user ids, post ids, thread ids.
 
Top