one.sh search & delete duplicate files

Source (link to git-repo or to original if based on someone elses unmodified work): Add the source-code for this project on opencode.net

Score 50.0%

Description:

A GUI/Dialog to search for duplicates of files. This check depends on MD5, not on file-name. It searchs all folders of an given path an deletes (if you wish) all duplicates and resulting empty folders. You can define a file-extension or search for * and so on in KDE-Dialog.

Everthing happens due to using this script is not under my responsibility.

Its useful to test it in a dir with copies of your files. Don't use it in program-folders - it's useful just for your Musik-, Picture-, Movie- or Documents-Folders.

Last changelog:

12 years ago

1.3.1-beta-testing NOT for USE!!!
1.3 Reduced Dialog-Windows and add a readme.
1.2 1st version
1.1 buggy ...

ToDo's:
- Cancel-Buttons in Progress.
- Copy or Symlink as option.
- Edit Del-List in Script. Edit during runtime with an editor is possible.
- Progress-Bar while counting - any idears?
- Rename/Move instead of delete.

dar38
10 years ago

Hello,
if anybody cares, I wrote a simple patch for editing the list of the files to be deleted. Here it is
[code]
--- one.sh 2009-01-05 11:52:50.000000000 +0100
+++ myone.sh 2009-01-05 11:52:50.000000000 +0100
@@ -114,6 +114,26 @@
echo "Duplicates: $u" >> ./diff.kjo
echo " " >> ./diff.kjo
kdialog --title "All multiple Files" --textbox diff.kjo 800 600
+
+
+ONEEDITOR=kate
+MODIFY=yes
+until [ $MODIFY = "no" ]; do
+FILENAME="loesch_liste.kjo"
+x=`kdialog --title "Keep just one!" --inputbox "Would you like to edit the list
?" "no"`
+if test "$x" = "yes"; then
+MODIFY="yes"
+kdialog --title "Keep just one!" --msgbox "Please edit the difference file.
+Be careful, don't forget to save the file after editting. "
+$ONEEDITOR $FILENAME
+else
+MODIFY="no"
+fi+
+
+
+
x=`kdialog --title "Keep just one!" --inputbox "Please enter yes to delete $u f
iles!" "no"`
if test "$x" != "yes"; then
kdialog --title "Keep just one!" --msgbox "Nothing happens to your files
"
[/code]
+done

Report

fireedo
12 years ago

this is exactly what I need :)
I have thousand mp3's files and a lot of them are duplicated and with your script I can find it without doin' "dirty" job by find it one by one....
thanx a lot

Report

deft
12 years ago

Thank's for your reply!

That's what i made it for. Having a song multiple times in amarok still drives me crazy ;)

I hope everthinks works well with my script on your machine :)

greetz from berlin - jens

Report

panzi
12 years ago

I wrote a similar tool a while ago. I used python and did this: I first used the file size as key. this is very fast and sorts many files out. then I made a sha1/md5 (chooseable) sum and used that as key and then the remaining collisions where binary compared (optinal).

the thing with the file size speeds the process up a lot! ;)

Report

deft
12 years ago

Yap, this makes sense! I'll try bring it in my script!

But my actual problem is: file name with spaces...

... and building links ... cause:

if i change the IFS to anything else than a "whitespace" DCOP failed with the gauge - ok - a gauge isnt'that necassary...

So, the next step is a "ln -s"-function.

a basicly working version is done - hopefully i bring it to a "testing"-state

Thx 4 ur reply - jens

Report

panzi
12 years ago

But my actual problem is: file name with spaces...

That was one of the reasons why I did choose python. ;)
That and pythons nice & fast dictionaries (hashtables).

If you want to take a look at it, here is my script: http://twoday.tuwien.ac.at/pub/stories/255295/

Report

deft
12 years ago

thx 4 the link, helpful 4 me ;)

Report

brcha
12 years ago

I agree with bsander that there are too many dialogs. But I guess that can be cut down easily.

One suggestion. Maybe it would be nice if you offered an option to symbolic link the files to the "master file" (what ever that is). If you search the .mp3s, it is ok to delete the dupes. But if you search your programming directories, it is probably smarter to symlink libtool.sh to some /usr/.../libtool.sh than to delete them all :)

Report

deft
12 years ago

Yap. That's what i meant with the Point "Link" in the ToDo.
My Idea of using the script was (at first) to reduce the size of my mp3-collection...
Set Symlinks instead of deleting makes sense... i'll try my very best - hopefully i'll find time for that.

Please excuse my bad english...

Report

bsander
12 years ago

if you need three screenshots to display the process, you really need to cut down on the dialogs..

Report

deft
12 years ago

hmm, it seem's that i'am infected with some kind of M$-Virus...confirming the confirmation of a confirmation. You're right. During my "Changing a shell-script to a KDE-Dialog" i used the mesages to "echo" the variables and "see" the startpoint of every function. It's all done by try and error...

Thanks for the hint. ;)

Report