• This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.

200000 files need... :(

Wuebit

Well-known member
#1
Ok so i have around 200000 image which i need to add to a folder but here is the problem i need single folder's named after the main file name. So say a few files named test.png test2.png test3.png

/home/some-random-folder/test.png,test2.png,test3.png etc... needs to be
/home/some-random-folder/test/test.png
/home/some-random-folder/test2/test2.png
/home/some-random-folder/test3/test3.png

I really don't want to download them all and use filemonkey to make the folders to reupload them, Is there a way to do this on my server? am using debian lenny

Cheers
 

Fred Sherman

Well-known member
#2
Not a problem. 200,000 files will take a while, but its a simple script.

From the command line in /home/some-random-folder/:

for FILE in `ls`
do
set -A DIR `echo $FILE | cut -d '.' -f 1`
mkdir $DIR
mv $FILE $DIR
done

Wait for a bit and the job is done. If you need the directories to have a specific permission, just add:

chmod 777 $DIR

after the mkdir command. 777 was just an example. Use the bitmask you need, if it all.
 

Wuebit

Well-known member
#3
Not a problem.

From the command line in /home/some-random-folder/:

for FILE in `ls`
do
set -A DIR `echo $FILE | cut -d '.' -f 1`
mkdir $DIR
mv $FILE $DIR
done
Thanks man <3
one question tho am not good with command's so ive cd/ to the folder


:/home/myanimeisland/public_html/booru/test# echo $FILE | cut -d '.' -f 1
translated
/home/myanimeisland/public_html/booru/test# mkdir $DIR
mkdir: missing operand
Try `mkdir --help' for more information.

I got this.

edit also i don't understand this bit "for FILE in `ls`" ;/
 

Fred Sherman

Well-known member
#4
Okay, this will make it easier. Unzip this file and upload it to /home/myanimeisland/public_html/booru/test

# chmod 755 ./file_images.ksh
# ./file_images.ksh&

Done. Wait for it to complete in the background. Do not close your ssh/telnet session until it is complete.
 

Attachments

Wuebit

Well-known member
#5
Okay, this will make it easier. Unzip this file and upload it to /home/myanimeisland/public_html/booru/test

# chmod 755 ./file_images.ksh
# ./file_images.ksh&

Done. Wait for it to complete in the background. Do not close your ssh/telnet session until it is complete.
:/home/myanimeisland/public_html/booru/test# ./file_images.ksh&
[1] 27937
:/home/myanimeisland/public_html/booru/test# -bash: ./file_images.ksh: /bin/ksh: bad interpreter: No such file or directory

Give this mate
i put the file in /home/myanimeisland/public_html/booru/test
chmod it to 755

and the 2nd command give me the line's above
Thanks alot btw^^
 

Wuebit

Well-known member
#7
type

# which ksh

What is the result?
Just goes to the next line mate
locate tho give's this

$ :/home/myanimeisland/public_html/booru/test# locate ksh
/home/myanimeisland/public_html/booru/contrib/simpletest/data/bedroom_workshop.jpg
/home/myanimeisland/public_html/booru/uploads/2girls against_wall blonde_hair blush bookshelf hat kirisame_marisa kiss long_hair multiple_girls patchouli_knowledge purple_hair rotan rough sweatdrop touhou yuri.jpg
/home/myanimeisland/public_html/booru/uploads/bad_id blonde_hair book bookshelf library long_hair name_(pixiv) original ponytail ribbon solo stone_wall.jpg
/home/myanimeisland/public_html/booru/uploads/book bookshelf candle chair crescent cup geshopu hair_ribbon hat library patchouli_knowledge purple_hair quill reading ribbon sitting solo table teacup teapot touhou voile.jpg
/home/myanimeisland/public_html/booru/uploads/book bookshelf crescent hair_ribbon hat head_wings koakuma necktie patchouli_knowledge plastic_eraser ribbon touhou voile.jpg
/usr/lib/ruby/gems/1.8/gems/tzinfo-0.3.24/lib/tzinfo/definitions/America/Danmarkshavn.rb
/usr/share/doc/libfcgi0ldbl/www5-api-workshop.html
/usr/share/man/man1/shtool-mkshadow.1.gz
/usr/share/man/pl/man1/ksh.1.gz
/usr/share/ri/1.8/system/Generators/CHMGenerator/check_for_html_help_workshop-i.yaml
/usr/share/shtool/sh.mkshadow
/usr/share/vim/vim71/doc/workshop.txt
/usr/share/zoneinfo/America/Danmarkshavn
/usr/share/zoneinfo/posix/America/Danmarkshavn
/usr/share/zoneinfo/right/America/Danmarkshavn
$ :/home/myanimeisland/public_html/booru/test#
 

Fred Sherman

Well-known member
#18
Here are several problems you have:

1. File names are too long
2. Filenames have spaces in them. Windows allows this, but Unix/Linux systems do not handle them well.
3. Non-alphanumeric characters in filenames. Some of these chaacters (*^$) have specials meanings to shells and editors. This can also cause problems.

I think you're going to find 100% automation isn't going to happen. At the end of the process, you're still going to end up with several hundred outliers that you'll need to address by hand.

The thing about image files is its a little like playing whisper down the lane. By the time you get it, it may have been renamed 1000 times. I wonder if they have any embedded metadata that could be extracted to get the original filename?
 

Wuebit

Well-known member
#19
Here are several problems you have:

1. File names are too long
2. Filenames have spaces in them. Windows allows this, but Unix/Linux systems do not handle them well.
3. Non-alphanumeric characters in filenames. Some of these chaacters (*^$) have specials meanings to shells and editors. This can also cause problems.

I think you're going to find 100% automation isn't going to happen. At the end of the process, you're still going to end up with several hundred outliers that you'll need to address by hand.

The thing about image files is its a little like playing whisper down the lane. By the time you get it, it may have been renamed 1000 times. I wonder if they have any embedded metadata that could be extracted to get the original filename?
K mate it's ok ill redownload them and trim them with filemonkey :) Thank again for the help mate