1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

200000 files need... :(

Discussion in 'Off Topic' started by Wuebit, Feb 17, 2011.

  1. Wuebit

    Wuebit Well-Known Member

    Ok so i have around 200000 image which i need to add to a folder but here is the problem i need single folder's named after the main file name. So say a few files named test.png test2.png test3.png

    /home/some-random-folder/test.png,test2.png,test3.png etc... needs to be

    I really don't want to download them all and use filemonkey to make the folders to reupload them, Is there a way to do this on my server? am using debian lenny

  2. Fred Sherman

    Fred Sherman Well-Known Member

    Not a problem. 200,000 files will take a while, but its a simple script.

    From the command line in /home/some-random-folder/:

    for FILE in `ls`
    set -A DIR `echo $FILE | cut -d '.' -f 1`
    mkdir $DIR
    mv $FILE $DIR

    Wait for a bit and the job is done. If you need the directories to have a specific permission, just add:

    chmod 777 $DIR

    after the mkdir command. 777 was just an example. Use the bitmask you need, if it all.
    Floris likes this.
  3. Wuebit

    Wuebit Well-Known Member

    Thanks man <3
    one question tho am not good with command's so ive cd/ to the folder

    :/home/myanimeisland/public_html/booru/test# echo $FILE | cut -d '.' -f 1
    /home/myanimeisland/public_html/booru/test# mkdir $DIR
    mkdir: missing operand
    Try `mkdir --help' for more information.

    I got this.

    edit also i don't understand this bit "for FILE in `ls`" ;/
  4. Fred Sherman

    Fred Sherman Well-Known Member

    Okay, this will make it easier. Unzip this file and upload it to /home/myanimeisland/public_html/booru/test

    # chmod 755 ./file_images.ksh
    # ./file_images.ksh&

    Done. Wait for it to complete in the background. Do not close your ssh/telnet session until it is complete.

    Attached Files:

    Vodkaholic likes this.
  5. Wuebit

    Wuebit Well-Known Member

    :/home/myanimeisland/public_html/booru/test# ./file_images.ksh&
    [1] 27937
    :/home/myanimeisland/public_html/booru/test# -bash: ./file_images.ksh: /bin/ksh: bad interpreter: No such file or directory

    Give this mate
    i put the file in /home/myanimeisland/public_html/booru/test
    chmod it to 755

    and the 2nd command give me the line's above
    Thanks alot btw^^
  6. Fred Sherman

    Fred Sherman Well-Known Member


    # which ksh

    What is the result?
  7. Wuebit

    Wuebit Well-Known Member

    Just goes to the next line mate
    locate tho give's this

    $ :/home/myanimeisland/public_html/booru/test# locate ksh
    /home/myanimeisland/public_html/booru/uploads/2girls against_wall blonde_hair blush bookshelf hat kirisame_marisa kiss long_hair multiple_girls patchouli_knowledge purple_hair rotan rough sweatdrop touhou yuri.jpg
    /home/myanimeisland/public_html/booru/uploads/bad_id blonde_hair book bookshelf library long_hair name_(pixiv) original ponytail ribbon solo stone_wall.jpg
    /home/myanimeisland/public_html/booru/uploads/book bookshelf candle chair crescent cup geshopu hair_ribbon hat library patchouli_knowledge purple_hair quill reading ribbon sitting solo table teacup teapot touhou voile.jpg
    /home/myanimeisland/public_html/booru/uploads/book bookshelf crescent hair_ribbon hat head_wings koakuma necktie patchouli_knowledge plastic_eraser ribbon touhou voile.jpg
    $ :/home/myanimeisland/public_html/booru/test#
  8. Fred Sherman

    Fred Sherman Well-Known Member

    Type this:

    # env | grep ^SHELL

    What does it come back with?
  9. Wuebit

    Wuebit Well-Known Member

  10. Fred Sherman

    Fred Sherman Well-Known Member

    Try this file instead. Same thing, rewritten for bash.

    Attached Files:

    Vodkaholic likes this.
  11. Floris

    Floris Guest

    <goes offtopic for a second and says: unix cli ftw!> $this->Sherman['thumbsup']++;
    Vodkaholic and Fred Sherman like this.
  12. Wuebit

    Wuebit Well-Known Member

    Close kind of worked

  13. Fred Sherman

    Fred Sherman Well-Known Member

    The filename (without extension) must be a valid directory name.
  14. Wuebit

    Wuebit Well-Known Member

  15. Wuebit

    Wuebit Well-Known Member

    Fred there is something else i need to do to some other file's. I need to trim(delete) the first 4 character's from a set of file's and 2 from some more file's can this be done?

  16. Fred Sherman

    Fred Sherman Well-Known Member

    Sure. Give me a sample of the filenames.
  17. Wuebit

    Wuebit Well-Known Member

  18. Fred Sherman

    Fred Sherman Well-Known Member

    Here are several problems you have:

    1. File names are too long
    2. Filenames have spaces in them. Windows allows this, but Unix/Linux systems do not handle them well.
    3. Non-alphanumeric characters in filenames. Some of these chaacters (*^$) have specials meanings to shells and editors. This can also cause problems.

    I think you're going to find 100% automation isn't going to happen. At the end of the process, you're still going to end up with several hundred outliers that you'll need to address by hand.

    The thing about image files is its a little like playing whisper down the lane. By the time you get it, it may have been renamed 1000 times. I wonder if they have any embedded metadata that could be extracted to get the original filename?
    Vodkaholic likes this.
  19. Wuebit

    Wuebit Well-Known Member

    K mate it's ok ill redownload them and trim them with filemonkey :) Thank again for the help mate

Share This Page