one.sh search & delete duplicate files

System Software

Source (link to git-repo or to original if based on someone elses unmodified work): Add the source-code for this project on opencode.net

0
5.0
Available as/for: -
Description:

A GUI/Dialog to search for duplicates of files. This check depends on MD5, not on file-name. It searchs all folders of an given path an deletes (if you wish) all duplicates and resulting empty folders. You can define a file-extension or search for * and so on in KDE-Dialog.

Everthing happens due to using this script is not under my responsibility.

Its useful to test it in a dir with copies of your files. Don't use it in program-folders - it's useful just for your Musik-, Picture-, Movie- or Documents-Folders.
Last changelog:

12 years ago

1.3.1-beta-testing NOT for USE!!!
1.3 Reduced Dialog-Windows and add a readme.
1.2 1st version
1.1 buggy ...

ToDo's:
- Cancel-Buttons in Progress.
- Copy or Symlink as option.
- Edit Del-List in Script. Edit during runtime with an editor is possible.
- Progress-Bar while counting - any idears?
- Rename/Move instead of delete.

11 comments
Comments/Ratings

Please login or register to add a comment or rating

dar38

12 years ago

Hello,
if anybody cares, I wrote a simple patch for editing the list of the files to be deleted. Here it is
[code]
--- one.sh 2009-01-05 11:52:50.000000000 +0100
+++ myone.sh 2009-01-05 11:52:50.000000000 +0100
@@ -114,6 +114,26 @@
echo "Duplicates: $u" >> ./diff.kjo
echo " " >> ./diff.kjo
kdialog --title "All multiple Files" --textbox diff.kjo 800 600
+
+
+ONEEDITOR=kate
+MODIFY=yes
+until [ $MODIFY = "no" ]; do
+FILENAME="loesch_liste.kjo"
+x=`kdialog --title "Keep just one!" --inputbox "Would you like to edit the list
?" "no"`
+if test "$x" = "yes"; then
+MODIFY="yes"
+kdialog --title "Keep just one!" --msgbox "Please edit the difference file.
+Be careful, don't forget to save the file after editting. "
+$ONEEDITOR $FILENAME
+else
+MODIFY="no"
+fi+
+
+
+
x=`kdialog --title "Keep just one!" --inputbox "Please enter yes to delete $u f
iles!" "no"`
if test "$x" != "yes"; then
kdialog --title "Keep just one!" --msgbox "Nothing happens to your files
"
[/code]
+done

Report

fireedo

13 years ago

this is exactly what I need :)
I have thousand mp3's files and a lot of them are duplicated and with your script I can find it without doin' "dirty" job by find it one by one....
thanx a lot

Report

C

deft

13 years ago

Thank's for your reply!

That's what i made it for. Having a song multiple times in amarok still drives me crazy ;)

I hope everthinks works well with my script on your machine :)

greetz from berlin - jens

Report

panzi

13 years ago

I wrote a similar tool a while ago. I used python and did this: I first used the file size as key. this is very fast and sorts many files out. then I made a sha1/md5 (chooseable) sum and used that as key and then the remaining collisions where binary compared (optinal).

the thing with the file size speeds the process up a lot! ;)

Report

C

deft

13 years ago

Yap, this makes sense! I'll try bring it in my script!

But my actual problem is: file name with spaces...

... and building links ... cause:

if i change the IFS to anything else than a "whitespace" DCOP failed with the gauge - ok - a gauge isnt'that necassary...

So, the next step is a "ln -s"-function.

a basicly working version is done - hopefully i bring it to a "testing"-state

Thx 4 ur reply - jens

Report

panzi

13 years ago

But my actual problem is: file name with spaces...

That was one of the reasons why I did choose python. ;)
That and pythons nice & fast dictionaries (hashtables).

If you want to take a look at it, here is my script: http://twoday.tuwien.ac.at/pub/stories/255295/

Report

C

deft

13 years ago

thx 4 the link, helpful 4 me ;)

Report

brcha

14 years ago

I agree with bsander that there are too many dialogs. But I guess that can be cut down easily.

One suggestion. Maybe it would be nice if you offered an option to symbolic link the files to the "master file" (what ever that is). If you search the .mp3s, it is ok to delete the dupes. But if you search your programming directories, it is probably smarter to symlink libtool.sh to some /usr/.../libtool.sh than to delete them all :)

Report

C

deft

14 years ago

Yap. That's what i meant with the Point "Link" in the ToDo.
My Idea of using the script was (at first) to reduce the size of my mp3-collection...
Set Symlinks instead of deleting makes sense... i'll try my very best - hopefully i'll find time for that.

Please excuse my bad english...

Report

bsander

14 years ago

if you need three screenshots to display the process, you really need to cut down on the dialogs..

Report

C

deft

14 years ago

hmm, it seem's that i'am infected with some kind of M$-Virus...confirming the confirmation of a confirmation. You're right. During my "Changing a shell-script to a KDE-Dialog" i used the mesages to "echo" the variables and "see" the startpoint of every function. It's all done by try and error...

Thanks for the hint. ;)

Report

12 years ago

1.3.1-beta-testing NOT for USE!!!
1.3 Reduced Dialog-Windows and add a readme.
1.2 1st version
1.1 buggy ...

ToDo's:
- Cancel-Buttons in Progress.
- Copy or Symlink as option.
- Edit Del-List in Script. Edit during runtime with an editor is possible.
- Progress-Bar while counting - any idears?
- Rename/Move instead of delete.

12345678910

0 comments
Comments

Please login or register to add a comment

Be the first to comment
File (click to download) Version Description PackagetypeArchitectureRelease ChannelDevices Downloads Date Filesize DL OCS-Install MD5SUM
*Needs pling-store or ocs-url to install things
product-maker
deft
Pling
0 Affiliates

Support-Box

Become a Supporter of "one.sh search & delete duplicate files" and similar System Software via a monthly donation of
  • $0.99
  • $2
  • $5
  • $10
  • $
    Enter a free Amount
one.sh search & delete duplicate files is part of Apps, Games and Add-ons, which is supported by the following people:
Other System Software:
product
product
product
product
product
product
Details
license
version
1.3
updated Nov 27 2008
added Jan 21 2007
downloads 24h
0
mediaviews 24h 0
pageviews 24h 0
Report Misuse Report SPAM
System Tags app software