Silk Road forums
Discussion => Security => Topic started by: DigitalAlch on June 20, 2011, 06:15 am
-
Scrubbing Meta Data from Images in Ubuntu
This is pretty vital. I have notice people are uploading images a lot, both for avatars and in posting items. Something you may not know is that most digital cameras add hidden data called EXIF data, and on top of that most computers add in there own metadata such as IPTC and XMP. This data can reveal alot of data you do not want others to have. As such we should scrub this data before uploading. You should all be accessing this website (and all of the DeepWeb) through a VirtualMachine running Ubuntu (I'll write a guide on that later).
So launch up Ubuntu, go to shell (Applications -> Accessory -> Terminal) and type:
sudo apt-get install exiv2(Press y then enter when asked)
This will install jhead, the program that will accomplish this. Once you have installed it you can either find out more info on it by typing:
man exiv2or just get down to businesses. To do this we will move to the directory our images are in:
cd Pictures(this moves to folder called pictures) IF you are wondering what folder are in your current directory you can type
lsNow we can either remove the meta data from all images:
exiv2 -d a *.jpgKeep in mind if you have any .JPG / .JPEG / .jpeg remember to run for each one. Example:
jexiv2 -d a *.JPEG
Peace,
DigitalAlch
-
How do you do this with Windows 7?
-
linux
exiv2 rm *.* for the entire picture folder
windows use
jpg scruber
http://jpeg-scrubber.com (CAUTION clear web alert)
-
linux
exiv2 rm *.* for the entire picture folder
windows use
jpg scruber
http://jpeg-scrubber.com (CAUTION clear web alert)
Nice little program.
-
thanks mate,
god ive learnt so much in the past few weeks,things i would never have thought of in a million years,in regards to security.
-
How do you do this with Windows 7?
I'm sure any number of exif utilities exist for Windows. It's just a matter of finding one you like. Alternatively a MacGyver method for any OS is to save it in a format that doesn't include exif data (BMP or PNG would be good choices), close it, open the new file, and save back to jpeg.
-
How do you do this with Windows 7?
I'm sure any number of exif utilities exist for Windows. It's just a matter of finding one you like. Alternatively a MacGyver method for any OS is to save it in a format that doesn't include exif data (BMP or PNG would be good choices), close it, open the new file, and save back to jpeg.
windows use
jpg scruber
http://jpeg-scrubber.com (CAUTION clear web alert)
-
Thank you for this guide!
-
+1
very nice tutorial digi
i do almost the exact same thing with my Ubuntu when using it on Tor.
-
Nice mate & thank u.
-
Great guide, thanks! 8)
-
Do .png files have such metadata?
-
You should all be accessing this website (and all of the DeepWeb) through a VirtualMachine running Ubuntu (I'll write a guide on that later).
Excellent guide, why not mention this: http://dee.su/liberte
-
Great tut. I was aware of Exif data but was unaware it was so easy to get rid of.
Looking forward to your VM tutorial. I currently do not use a VM because I am concerned about how it will affect my machine performance. I suppose it is time to try it out.
-
I dunno if this removes all the metadata or not but in Windows 7 you can right click on the image -> Properties -> Details -> Remove properties and information (way at the bottom of the dialog box)
-
Or you can save the image as PNG. PNG doesn't support any sort of EXIF data.
-
Or you can save the image as PNG. PNG doesn't support any sort of EXIF data.
Careful with this. PNG does not officially support EXIF data, but the PNG specification absolutely allows metadata to be stored in an image. Whether or not this is common practice, I don't know, but please be safe.
For more information (really nerdy), see the PNG spec at: http://www.w3.org/TR/PNG/ (Warning: clearweb). Relevant section 11.3.4.
-
linux
exiv2 rm *.* for the entire picture folder
windows use
jpg scruber
http://jpeg-scrubber.com (CAUTION clear web alert)
Thank you so much for the universal scrubber!
-
I use an image upload site called anony.ws. They scrub all images of exif data after uploaded :)
-DF
drugfather
-
wow excellent guide. I am so happy that I read this post. I learned a lot from this guide and I really want to thank the author.
-
Thanks alot digital. Silk Road is lucky to have someone like you here to help all of us nubs! I am already putting this guide to good use.
-
I found a portable application that lets you drag the file to it window and it strips the information.
hxxp://www.pendriveapps.com/jpeg-and-png-stripper-remove-metadata/
-
does jpeg scrubber erase metadata from a pic?
-
This should help some of the google illiterate members...
-
Noob question.
I downloaded GPGtools for apple.
How do I access the clipboard where I can type messages to encrypt?
The GPG program only has "New, Import, Export" as options.
Thanks.
-
Do GIFs have metadata? I knew jpgs did but never even wondered if any other picture file could... duh!
Thanks!
-
What does "(CAUTION clear web alert)" mean please? And what is "Ubuntu"?
Thanx,
Oz
linux
exiv2 rm *.* for the entire picture folder
windows use
jpg scruber
http://jpeg-scrubber.com (CAUTION clear web alert)
-
After about 10 minutes waiting for 'http://dee.su/liberte' to load it said, "To use this page you need to enable JavaScript in your web browser" Am I missing something AGAIN or do I enable scripting? I thought that was a no-no.
Oz
You should all be accessing this website (and all of the DeepWeb) through a VirtualMachine running Ubuntu (I'll write a guide on that later).
Excellent guide, why not mention this: http://dee.su/liberte
-
For those 'other' indigent members; Jpeg Scrubber is NOT free. It doesn't say that at the download site either. So you don't waste a bunch of time downloading something you may have thought was 'free' - it is not. Thanx for you time - Oz
-
I've just had my first interest in posting any picture, here or on SR.
I use Android exclusively. This idea of images having revealing info has now become necessary for me. What picture cleaning is there for Android? Is their a site I can or must use? Someone mentioned anony.ws. Any good?
Edit: Forums doesn't allow me to browse my Android. Maybe administration can make forums smart phone friendly?
-
I've just had my first interest in posting any picture, here or on SR.
I use Android exclusively. This idea of images having revealing info has now become necessary for me. What picture cleaning is there for Android? Is their a site I can or must use? Someone mentioned anony.ws. Any good?
Android is Linux. Just install a terminal emulator from the market.
-
great good to know if i upload photos
-
What I have done in the past is take a screen shot of my picture and then paste the screen shot and re-save it. Now I am wondering if this is good enough, or does Windows put data in files saved in paint?
Thx for the guide!
-
Thanks for the heads up OP. I believe everything went OK but I think you have a typo in your last code.It says "jexiv2" and I spent a good time looking for a package that didn't exist before I realized that it was typo. Especially since it makes it look like it is a version of exiv2 for JPEG files, And that was what that last code was pertaining too.
I could see that typo giving a lot of ubuntu noobs a headache.
-
Very nice post. Actually thought it had no sense to remove the EXIF data (like shutter speed, ISO, aperture, etc). But it contains also the camera model used, that may be helpful, if you own this camera and its found in your basement by cops - right?
-
Very helpful topic, thanks :)
-
Not sure if it has been mentioned yet, but jhead is also good and may even come with ubuntu already. If you have it installed you can do a whole directory at a time by going to the directory with the images and doing
ls | xargs jhead -purejpeg
That's it, and it's done. You could even include sub directories by changing the ls command with find
find . -type f -iname *.jp*g -exec jhead -purejpg {} \;
*Edit* I just ran this command and got an error, the 'e' in jpeg was bad. I always do that. That's why I use jp*g when I'm working with a load of jpgs. Also the -type 'file' was wrong, it's -type f . I didn't make the same mistake when I just used it, but saw it when I came back here to fix the jp*g shit. Using the -exec flag with find, you can really make some awesome one-liners, and that's what makes *nix so awesome imo. If I thought there was any interest at all in explaining how that command works I'd share, but I seriously doubt anyone gives a fuck about how some command is read. Sorry for being high and trying to add useful content. I'll go back to adding nothing of value now, thank you for your time.
Either way, this is the easiest way on earth to clean 3000 jpgs of metadata
-
would be nice if SR would remove exif information automatically!
-
Not sure if it has been mentioned yet, but jhead is also good and may even come with ubuntu already. If you have it installed you can do a whole directory at a time by going to the directory with the images and doing
ls | xargs jhead -purejpeg
That's it, and it's done. You could even include sub directories by changing the ls command with find
find . -type f -iname *.jp*g -exec jhead -purejpg {} \;
*Edit* I just ran this command and got an error, the 'e' in jpeg was bad. I always do that. That's why I use jp*g when I'm working with a load of jpgs. Also the -type 'file' was wrong, it's -type f . I didn't make the same mistake when I just used it, but saw it when I came back here to fix the jp*g shit. Using the -exec flag with find, you can really make some awesome one-liners, and that's what makes *nix so awesome imo. If I thought there was any interest at all in explaining how that command works I'd share, but I seriously doubt anyone gives a fuck about how some command is read. Sorry for being high and trying to add useful content. I'll go back to adding nothing of value now, thank you for your time.
Either way, this is the easiest way on earth to clean 3000 jpgs of metadata
Addendum: jhead is available for OSX users through macports as well. Thank you. :)
-
Just saw that I also put "jpEg" in the -purejpg tag in the first example lol. I have not been able to shake that habit no matter hard I try I still catch myself doing it.
That's good news about jhead being available for mac, because it really is a nice, clean, small solution for washing jpgs and now more people will find and use it.
-
I downloaded a photo of some MDMA pills some vendor had taken. The meta info was all there. It was taken on a Blackberry - my expertise is limited but I am sure some probing would reveal the serial and that is likely linked to the phone account which reveals the person who took the photo! (this is what the cops would think)
I guess I ought to warn that vendor.
I get rid of meta if by simply displaying my photo then taking a screenshot! That screenshot has meta info but I doubt I can be traced from it as I will do this using virtual box. Not sure if this is 100% proof safe but a screen shot will not carry any of the photo meta info. Maybe the operating system and serial number - the same used by millions!
I do not use Linux much but have got to work out my new Linux set up and get my wireless working!!
Thanks for this read - always good to see security issues taken seriously.
A new generation has been raised to accept technology as second nature. People my age suspect it has good and bad uses. The bad being tracking us all!!
So be careful out there! :)
-
I dunno if this removes all the metadata or not but in Windows 7 you can right click on the image -> Properties -> Details -> Remove properties and information (way at the bottom of the dialog box)
When I do this it clears all the attributes from iPhone photos EXCEPT for the GPS... probably the most important one to clear. So watch out. And honestly if this is the case go on your iPhone and disable location so its not stored to begin with.
-
Thank you sir!
-
I don't mean to hijack this thread or anything...... but if you are reading this thread than you are most likely an ubuntu user and probably a new one at that. So I would like to say that if ANYBODY has any questions about this OS, I would be happy to help you in any way I can and I encourage you to give me a PM.
I offer this service freely because I feel that I owe a debt to SR for being such a wonderful community and this is one of the many subjects that I excel in.
-
Tinfoilhattery follows.
Please be aware of camera fingerprinting.
It is possible to associate photos with each other through unique noise pattern deviations in CCD sensors.[1]
I can't give reliable advice on how to avoid this but ideally you would never use the same camera for anything that could be connected to your identity. If that's not practical, at least always make sure to resize the image and maybe add some noise in a photo editing program.
It is not known whether LEAs use CCD noise fingerprinting but given that it can be fully automated, it's not far fetched to assume they do. I'd be damned if there isn't an NSA server cluster somewhere that archives and analyzes images, particularly from social networks like Facebook. The archives only gain in value as analysis techniques are refined over the years.
I would not overestimate this method.
Just be careful.
==== CLEARNET ====
http://cs.iupui.edu/~tuceryan/pdf-repository/Lukas2006.pdf
================
P.S.: Besides CCD noise ID, which is publicly researched, there is the possibility that digital cameras add unique watermarks to images. That, however, is pure speculation based on the fact that many printer vendors are working with LEAs. Many printers add nearly invisible watermarks to every printed page. This is public information.
==== CLEARNET ====
https://www.eff.org/issues/printers
http://dkn255hz262ypmii.onion/index.php?topic=10719.0 (Your Laser Printer Will Testify Against You)
================
Bottom line: If you want to be better safe than sorry, use a separate camera.
-
what type information can MetaData reveal? ive heard sories of some phones leaving watermarks on pictures of the GPS co-ordinates of the location the picture was taken. not sure if its true
-
what type information can MetaData reveal? ive heard sories of some phones leaving watermarks on pictures of the GPS co-ordinates of the location the picture was taken. not sure if its true
Go install the exif viewer firefox addon and view the EXIF data on random images on the web. You will be shocked at what is recorded. In terms of GPS, almost every smart phone records the exact time and GPS (if its enabled) location for each picture. If GPS is off, it will even triangulate the position as an estimate. 4chan is an interesting place to EXIF view images LOL. It should be important to note that sites like IMGUR remove all EXIF data when a photo is uploaded.
-
It should be noted that PNG files often DO contain metadata. To see this metadata in Ubuntu, install the package pngmeta ("sudo apt-get install pngmeta", or search for it in a package manager). For example, files saved with GIMP by default have at least:
Comment: Created with GIMP
Modification Time: 01 Jan 1970 00:00:00 +0000
Albeit with the real time of creation, rather than the UNIX epoch. In GIMP at least, you can switch off the saving of this metadata in the options box that appears when you save a PNG file.
-
Yeah this is great. Sure someone has posted on here about this, but what we used to do is literally save a camera file/picture in a non-exif inclusive format like .png and send that or resave as a jpeg or gif afterwards. All exif sheds off.
Thanks !!
-
The Ubuntu solution is awesome. If you find yourself not being able to move away from Win7, or use a Mac, ExifTool is a great stripper too:
[Warning - clearweb link]
www.sno.phy.queensu.ca/~phil/exiftool
There's really no practical metadata it can't or won't strip out, and it's worked on data coming from every model of pro/con/crap -sumer camera I've owned as well as pics gathered from the internet. It's pretty awesome to use in the field, too, as long as you're not too squeemish about using a command line.
-
There is the metadata anonymization toolkit as well. https://mat.boum.org/
-
So launch up Ubuntu, go to shell (Applications -> Accessory -> Terminal) and type:
sudo apt-get install exiv2(Press y then enter when asked)
This will install jhead, the program that will accomplish this. Once you have installed it you can either find out more info on it by typing:
I don't know if I misunderstood something, or there is a mistake in the post here.
There seem to be two programs, exiv2 and jhead (by the description, it looks like both do the same thing). The command below should install exiv2, not jhead, as stated in the OP.
sudo apt-get install exiv2
Which one should I use?
-
The Ubuntu solution is awesome. If you find yourself not being able to move away from Win7, or use a Mac, ExifTool is a great stripper too:
[Warning - clearweb link]
www.sno.phy.queensu.ca/~phil/exiftool
There's really no practical metadata it can't or won't strip out, and it's worked on data coming from every model of pro/con/crap -sumer camera I've owned as well as pics gathered from the internet. It's pretty awesome to use in the field, too, as long as you're not too squeemish about using a command line.
Very cool, thankyou! +1
-
There is the metadata anonymization toolkit as well. https://mat.boum.org/
Great!
I've been planning this very same concept a few months ago,
unaware that a tool like this already exists.
Back then I had to work with Microsoft Office. Microsoft Word saves a TON of hidden meta-data.
This is perfect.
-
If i were to do a prnt scrn of an image, would this remove any metadata from the original image file?
-
Noob question.
I downloaded GPGtools for apple.
How do I access the clipboard where I can type messages to encrypt?
The GPG program only has "New, Import, Export" as options.
Thanks.
If you have Leopard 10.5.8 like me all you can do is look at your keys you made, you cant even import a pub key. You must do it through the terminal command line. Took me awhile to learn how to as well. LouisCypher is the SHIT.
-
Still not sure how to scrub pics yet. I have to encrypt, decrypt, and import keys all through line code. I really need to be able to ad pics.
-
Noob question.
I downloaded GPGtools for apple.
How do I access the clipboard where I can type messages to encrypt?
The GPG program only has "New, Import, Export" as options.
Thanks.
If you have Leopard 10.5.8 like me all you can do is look at your keys you made, you cant even import a pub key. You must do it through the terminal command line. Took me awhile to learn how to as well. LouisCypher is the SHIT.
Cheers.
I found an old GUI from the predecessor to GPGTools which will work on Leopard. It doesn't have the same features as GPGTools on Snow Leopard and above, but it lets you do basic tasks in a simple GUI. It's called GPGDropThing and I posted about it here:
http://dkn255hz262ypmii.onion/index.php?topic=35310.0
That said, you'll be better off in the long run by sticking with the command line.
I also now have a stickied thread in the Security forum with all my GPG posts listed:
http://dkn255hz262ypmii.onion/index.php?topic=38861.0
-
I am using the Imagemagic package to convert then from the camera size 3006x3496 or some size like that and then using jhead to strip them
both packages can be found on the standard linux repos.
I then wrote a script to handle the conversion.
#!/bin/sh
for i in `ls *.JPG`
do
convert $i -resize 800x600 `echo "$i" | cut -d. -f1`-s.jpg
done
jhead -purejpg *.JPG
jhead -purejpg *.jpg
But if you want it in a GUI you could always do this (KDE Interface)
#!/bin/sh
#
File=`kdialog --getopenfilename ~ "*.jpg||*.JPG"`
convert $File -resize 800x600 `echo "$File" | cut -d. -f1`-s.jpg
jhead -purejpg `echo "$File" | cut -d. -f1`-s.jpg
#
-
Can't you just upload to, say, imgur and then download it again? Imgur removes all metadata...
-
I mean call me studid but if my file pic name is cutedog how do I swipe this pic? Do I have to rename it?
-
Rename it? Why? If you want to get rid of the metadata you can use command line programs for linux that have been listed, or any of the other GUI applications that have been mentioned. If that still doesn't work you can honestly upload to imgur and download it again while on Tor. Photoshop even has an option to save images without any metadata. And as Crooked mentioned, you can even take a screenshot of the image and it will remove the metadata. It is not stored in the image data, it's just a header on the file so it has nothing to do with the filename.