Silk Road forums
Discussion => Security => Topic started by: luvaluva on October 18, 2012, 03:03 am
-
Today, I did a cursory web search, and a BUNCH of forum results from this site came up through onion.to and tor2web.org. Shouldn't onion results be free from crawling?
I haven't done a similar test for the main site, but would be happy to do such a search if anyone's interested. Either way, I think this may need to be fixed.
-
it isn't possible to fix it
-
it isn't possible to fix it
Which is why people need to be discreet. Loose lips sink ships, and all that.
-
it isn't possible to fix it
What if the forum is behind a members-only login, like the main site?
-
it isn't possible to fix it
yes it is.. make the forum read only by members, crawlers cnt login, this is why the main page is safe.
-
It is possible to make the forum only readable by members only, thus you wouldn't be able to view the forum as a guest, you would HAVE to sign up and some people might not like that..
-
and some people might not like that..
Fuck 'em. This is a security issue that's more important. People have been doxed on this site, and that can show up in Google searches. It would be way better for the community if the forum couldn't be crawled.
how?
-
omg stop the panic, admins can just set a robots.txt that forbids google to crawl.
LE can crawl the forum regardless of google and it comes with a built in search function anyway.
The more concerning question is how much tor bandwidth is used up by clearnet search engines crawling onionland.
On the other hand this might get a lot of people aware of Tor if they accidently stumble upon a .onion.to page and wonder about the strange url...
-
omg stop the panic, admins can just set a robots.txt that forbids google to crawl.
LE can crawl the forum regardless of google and it comes with a built in search function anyway.
The more concerning question is how much tor bandwidth is used up by clearnet search engines crawling onionland.
On the other hand this might get a lot of people aware of Tor if they accidently stumble upon a .onion.to page and wonder about the strange url...
Admin can't do that because the robots are not crawling SR they are crawling web based proxies that let people (insecurely) view .onion sites without Tor. The admin of those sites would need to add the robots.txt file , and this is why I say the issue is not fixable. Making it members only for login would not really fix the issue in the sense I was thinking (ie: I was thinking a robots.txt type of fix the situation) but it would make it so the robots get stuck on the login page instead of spider the forum.
That said, who gives a fuck. Loose lips sink ships hahahaha. This site has been all over the news of the entire god damn world and I don't keep track of the number of users but it must be over one hundred thousand by now. You guys sound fucking retarded to be tripping out about google having links to this thirty some thousand member public forum. Oh noez SR is fucked for sure now!!!1111
-
omg stop the panic, admins can just set a robots.txt that forbids google to crawl.
LE can crawl the forum regardless of google and it comes with a built in search function anyway.
The more concerning question is how much tor bandwidth is used up by clearnet search engines crawling onionland.
On the other hand this might get a lot of people aware of Tor if they accidently stumble upon a .onion.to page and wonder about the strange url...
Admin can't do that because the robots are not crawling SR they are crawling web based proxies that let people (insecurely) view .onion sites without Tor. The admin of those sites would need to add the robots.txt file , and this is why I say the issue is not fixable. Making it members only for login would not really fix the issue in the sense I was thinking (ie: I was thinking a robots.txt type of fix the situation) but it would make it so the robots get stuck on the login page instead of spider the forum.
That said, who gives a fuck. Loose lips sink ships hahahaha. This site has been all over the news of the entire god damn world and I don't keep track of the number of users but it must be over one hundred thousand by now. You guys sound fucking retarded to be tripping out about google having links to this thirty some thousand member public forum. Oh noez SR is fucked for sure now!!!1111
When I said "Loose lips sink ships" I was thinking not of the site, but of individual users on the site. The more discreet you are about your dealings, the better. Having the details of one's activities splashed all over Google is not a particularly good idea. That does double when people are sloppy about their personal security.
-
Doxing bypasses your personal discretion and safety measures.
Doxing has shit all to do with this
-
omg stop the panic, admins can just set a robots.txt that forbids google to crawl.
LE can crawl the forum regardless of google and it comes with a built in search function anyway.
The more concerning question is how much tor bandwidth is used up by clearnet search engines crawling onionland.
On the other hand this might get a lot of people aware of Tor if they accidently stumble upon a .onion.to page and wonder about the strange url...
Admin can't do that because the robots are not crawling SR they are crawling web based proxies that let people (insecurely) view .onion sites without Tor. The admin of those sites would need to add the robots.txt file , and this is why I say the issue is not fixable. Making it members only for login would not really fix the issue in the sense I was thinking (ie: I was thinking a robots.txt type of fix the situation) but it would make it so the robots get stuck on the login page instead of spider the forum.
That said, who gives a fuck. Loose lips sink ships hahahaha. This site has been all over the news of the entire god damn world and I don't keep track of the number of users but it must be over one hundred thousand by now. You guys sound fucking retarded to be tripping out about google having links to this thirty some thousand member public forum. Oh noez SR is fucked for sure now!!!1111
wouldnt the proxys that make SR & the forum visible in clearnet also forward on the robots.txt at the root of the webpage? And therefore the robots.txt would still work?
-
You are right actually, I was thinking if the domain is dkn255hz262ypmii.onion.to that it would check onion.to/robots.txt but it takes the subdomain into consideration so it would load http://dkn255hz262ypmii.onion.to/robots.txt /me embarrassed
-
My concern isn't with people finding out about this site, it's with the information that people reveal on this forum due to a false sense of security. The information people are revealing can lead to personal identification, especially if users of this site use the same username across multiple networks & public forums, where they may be more likely to reveal even more information.
I've been here for over a year -- I've watched the community grow from just a couple hundred members to the size it is now. Security is my main concern, and I think it should be a big concern for a lot of people here.
robots.txt would be a good start, and I believe that putting this behind a protected login should be a minimal step taken by administrators. That way, spiders that don't respect robots.txt will be stopped, and this site will fall off of the indices.
-
The security risk here is not software or google related.
It's human related.
This is a public forum. It's that simple.
-
The security risk here is not software or google related.
It's human related.
This is a public forum. It's that simple.
My point is exactly that the problem is a human problem. As a result, the software should be implemented in such a manner as to protect the people who may not know better.
-
Anything you post here is open for all the world to see. It's not just you and some fellow drug buyers/sellers. Whatever you post here is public. The police, journalists and entire internet are lurking.
It's good that Google makes you feel uncomfortable. It should.
-
Admin can't do that because....web based proxies...The admin of those sites would need to add the robots.txt file , and this is why I say the issue is not fixable.
BULLSHIT.
The forum admins can easily detect the Proxy IP range and BAN that IP range from viewing/spidering the site. Problem instantly solved.