Silk Road forums

So I talked with some computer security experts regarding this matter. I hold all of them in as high of regard as Theo, they are true experts . In general, they all seemed to agree with what Theo was saying about the inherent security issues related to using full hardware virtualization, however they did not in general come to the same conclusion as he did (that you are better off to not isolate firefox from external IP address than to use full hardware virtualization to do it).

The summary boils down to this: using full hardware virtualization does (very likely) have some security consequences, which could be significant or could be minor. I need to do more research to fully grasp the security implications of using full hardware virtualization, but the people I talked with have at least decent understandings of the issues. Many of the people I talked with suggested using full hardware virtualization to isolate network facing applications from Tor, saying that the security benefits this certainly brings will likely outweigh the security issues involved with full hardware virtualization. However, parvirtualization such as Xen offers the same isolation benefits as full hardware virtualization does, and it indeed does not have as much potential (probable?) security risk associated with it.

If the user is able to configure and use a paravirtualization system for isolation, they should use it instead of full hardware virtualization. Additionally, OS virtualization such as jails, zones etc offer much of the same isolation benefits without any of the risk associated with hardware virtualization (full hardware or paravirtualization). OS virtualization is probably easier for an attacker to break out of than hardware virtualization (either full or paravirtualization) but it avoids the risks that Theo is talking about with architecture virtualization security issues. Regarding my hacker friend who had previously told me that hardware virtualization offers stronger isolation, I misunderstood what he said because I didn't have a good enough understanding of virtualization when he was talking about it. Paravirtualization is also hardware virtualization, just not full hardware virtualization. full hardware virtualization probably is harder to break out of than OS virtualization, but paravirtualization is also probably harder to break out of and it greatly reduces potential security risks that are present with full hardware virtualization (due to the architecture issues that Theo was discussing).

These virtualization techniques can be layered, however using full hardware virtualization may not be a good idea if you are already using paravirtualization / os virtualization for isolation, because full hardware virtualization may have implementation flaws (including in the hardware that supports it) that could be used as an attack vector. This potential attack vector will not be present if you are not using full hardware virtualization.

So in short, if you can only be bothered to use the easy to configure full hardware virtualization solutions like Virtualbox, you should still use them to isolate network facing applications, instead of not isolating network facing applications at all. However, if you can configure the same thing using paravirtualization or OS virtualization you should use them instead and probably should avoid using full hardware virtualization as an additional layer. However, it is possible that using full hardware virtualization as an additional layer will add more security. It also may make you more insecure than only using OS virtualization or paravirtualization. However, attackers who can penetrate one layer of virtualization based isolation can likely penetrate other layers, using more than one layer may technically increase the 'depth' of the isolation but it will likely only be slowing an attacker down if they are capable of breaking out of isolation at all. Also, using full hardware virtualization may give attackers an additional vector that they otherwise would not have.

The general theme I saw was "many attackers can not break out of virtualized isolation, however the attackers who can break out of virtualized isolation can probably break out of as many layers of it as you use". And using full hardware virtualization does have potential/probable negative security implications that are not as much of an issue with paravirtualization and I don't think are issues at all with OS virtualization. But the risks of using full hardware virtualization do not out weigh the benefits of isolating Firefox from your external IP address, so if you don't plan to use anything else you should still be using full hardware virtualization based isolation . But there are safer ways to isolate Firefox from external IP address than using full hardware virtualization that should be used instead, if you have the skill to configure them.

Also, if you have enough extra machines it is probably more secure to use them to isolate firefox from the external IP address than to use a virtualization based solution. However, this doesn't mean you shouldn't use virtualization to do the same thing if you do not have extra machines or the skill required to isolate Firefox in this way.


here are some quotes:

"It is clear that you should try to keep network facing applications away from your external IP address"

"Using full hardware virtualization is better than not isolating Firefox, and it is much easier to configure than anything Theo would approve of"

"If they want to argue that full hardware virtualization is worse than not isolating Firefox away from your external IP address, ask them for an exploit that works against a full hardware virtualized system and not on an unisolated Firefox. They will not have one."

"Or ask for an easy to configure Theo approved solution for keeping Firefox isolated from the external IP address."

"One of the main things to keep in mind is that it is much easier to configure full hardware virtualization isolation than to configure any of the potentially better systems"

So there are some quotes from other people who are all security experts of the same caliber as Theo. Theo is also a security expert. They have different opinions. My personal feelings are that you should use paravirtualization or OS virtualization to isolate network facing applications from external IP address. If you can't be bothered to use paravirtualization or OS virtualization, you should still be using the easy to configure full hardware virtualization solutions. If you should use full hardware virtualization as an additional layer on top of OS virtualization or not is open for debate, it certainly has highly probable negative security implications, but it will also add an additional layer of isolation.

I hope that this helps to clear things up, or at least show another perspective on the issue. In the end nobody I talked with really disagreed with anything Theo said, they just think the risks of not isolating network facing applications from external IP address outweigh the risks of using full hardware virtualization to do so. Of course, they also all think that there are better solutions than using full hardware virtualization, they are just much more difficult to implement.

Another thing I would like to mention is that virtualization being used to isolate applications from external IP address has in practice prevented at least one hidden service from being traced after the feds pwnt it. I don't know the sort of virtualization that was used, but I think this is a clear example that there are serious benefits to using *some* sort of virtualization to isolate network facing applications from external IP address.

Also I still have not fully ruled out the possibility that SR is run by a federal agency for the purposes of intelligence gathering, however I am leaning against this idea because of the technologies used and the techniques suggested. As long as SR allows Tor and GPG and Bitcoin I think it is trust worthy, although to be fair almost all of the users here are using bitcoin incorrectly and can probably be deanonymized by financial network analysis. DEA might have realized that more people would use bitcoin incorrectly than correctly. However security is in the hands of the user no matter what. It is also possible the FBI or another agency found a zero day exploit in Tor or some other program and decided to launch a massive intelligence gathering operation against the drug using and dealing community (ability to identify 100k drug users and dealers around the world would be a world record setting law enforcement intelligence operation...people would be put in the law enforcement hall of fame). I know of one Tor zero day vulnerability and there is some research team that is claiming to have another although I think they have made extremely overstated claims in the past and are not being taken very seriously as they have not yet disclosed it in technical detail.
I don't think LE are smart enough to do these things though, although users on SR really need to start anonymizing their bitcoins and not by using SR's mix either.

like a chrooted Tor daemon

It is funny that you say this because virtualized solutions are widely recognized as more secure isolation than chroot. On most operating systems chroot ships with known jail breaking vulnerabilities. I hope you don't scare people away from using security enhancing solutions into using less secure solutions simply by quoting the opinion of one expert who just happens to be rabidly against almost any form of security that doesn't come from having absolutely perfect code. The fact that you suggest using chroot over using a paravirtualized system seems suspicious to me because it is widely thought that chroot is flawed and paravirtualized solutions offer stronger protection.

Also I am not certain if you can use chroot by itself to isolate firefox from external IP address, but I know you can use paravirtualization and full hardware virtualization to accomplish this goal. OpenBSD has a more secure version of chroot than most operating systems but it is even recognized that the OpenBSD version of chroot has inherent issues which are not present if you use paravirtualization or full hardware virtualization. I will wait for a detailed technical reply from Theo regarding the inherent security risks of using full hardware virtualization before I come to my final conclusion on the matter, but even if he talks poorly about paravirtualization (which avoids many of the issues he mentions when he talks specifically about the full hardware virtualization solution virtualbox, in the quotes you misconstrue as discussing virtualization based isolation in general) I will still be using it because I know it is a technique suggested by a number of very skilled security researchers, with Theo being (potentially) opposed to it but also not being the final authority regarding computer security.

For example, Qubes gains much of its security by automatically putting every launched application in an isolated virtualized environment, and FreeBSD ships with jails which also allow you to isolate applications with a layer of virtualization. Also Open Solaris has built in support for virtualization for security, Open Solaris Zones. Qubes, Jails and zones use virtualization that doesn't virtualize all of the hardware. Also Inferno uses virtual machines although I am not sure which type, I don't know much about inferno.

Again, I need to hear specific security risks from Theo before I come to my final conclusion on using full hardware virtualization, but right now I am leaning against it and towards using paravirtualization / OS virtualization solutions simply from what I have heard (from Theo, from other hackers who suggest not using Virtualbox or VMware due to code complexity and incorrectness of the hypervisor) and from what I have seen (Qubes, Open Solaris and FreeBSD use non-full hardware virtualization based isolation, although I am not sure if this is for security or for the other benefits it brings). I am still not convinced that it is actually a security risk to use full hardware virtualization systems, and I doubt I will ever be convinced against using virtualization based isolation at all.

vnconfig and virtual drives are essentially the same as using a VM for what we want (conceal your IP if anybody breaks in) except the difference is this virtual partition isn't being booted over buggy emulated hardware that hasn't been security audited whatsoever. vnconfig is integrated into OpenBSD and has been audited. Should an attacker breach tor somehow, they're now stuck on an encrypted virtual disk, on a separate partition in a chroot behind a firewall/NAT ect. so they'll only ever see internal IP addresses should they break out of chroot (which is incredibly easy to do.. chroot basically prevents applications from overloading mem and other problems, it's not really used for 'security' per se as anybody getting into a chroot can break out of it in a few easy steps according to these developers http://kerneltrap.org/Linux/Abusing_chroot)

Tor knows your real IP address so if they pwn Tor they have located you. If they can pwn Firefox they can configure it to go around Tor or use other techniques to determine your IP address, unless you take active measures against this happening. I don't think chrooting Firefox, by itself, can be used to prevent this. Explain in technical detail how to isolate an attacker who pwns firefox from being able to determine your external IP address by using vnconfig/chroot because I currently don't know how to do this. I will look over your solution and have some friends look it over as well. I am not saying it wont work, I just am not sure how to do it, or if it will work. I do know you can run Tor on a dedicated machine and firefox on another machine with only an internal IP address though, and I think this should be able to give isolation as well.

Yes chroot on most OS is easy to break out of but OpenBSD has a more secure version. However, most security people I talk with suggest using virtualization techniques for isolation over chroot because of how historically insecure chroot is.

After experimenting around with vnconfig making encrypted virtual disks and secure partitions I think I've managed to set up a pretty bulletproof tor session should the worst case scenario happen: NSA or Chineez government cracks/hax00rs tor, somehow the feds learn this method and use it to round us all up. Or worse my competitors find out this method and come after me :X

This indicates to me that you don't know what you are talking about, because it isn't Tor that you should be isolating so much as it is your network facing applications. If Tor itself is pwnt you are fucked no matter how many layers of isolation you are using.

and I can't believe I wasted so much time using X.

I can agree with this to an extent, I use command line for most things now. I only use command line for servers.

Loïc Duflot an expert in smashing out of X has basically said for a decade that X is not secure whatsoever no matter what tricks you do to isolate it.

X is insecure but there are various isolation tricks you can use. Qubes offers x isolation. You can also get your own x isolation by using virtualization. BTW OpenBSD with x windows is just as insecure as any other OS, if a single application is pwnt the attacker can EOP to root because there is NO GUI ISOLATION WITH X WINDOWS. The attacker can spy on ALL of your keystrokes and steal your password when you SU to root in a windowed terminal. This can be fixed by using virtualization. I wonder if Theo is aware of this.

Problem with computer security is the people that actually really know about it ....and allow misinformation like using Virtual Machines as a security barrier to exist unchallenged

This is cherry picking. There are numerous security experts who are strong advocates of using virtual machines for isolation. I am not certain if they would advocate the use of full hardware virtualization though (I will ask, and I also eagerly await some technical details from Theo explaining how full hardware virtualization is not only completely worthless but also horribly insecure), but I know the FreeBSD devs and the Qubes devs and the Open Solaris devs and the SEL4 devs are advocates of using virtualization for isolation. So are many other security experts. You have a biased opinion on virtualization based isolation if you are basing your opinion only on what the OpenBSD devs say. They are very against virtualization and imo seem to be against security via isolation in general, instead hoping that you use *only* code that has been audited by them for the past decade, because *only they* know how to write anything that isn't totally insecure shit.

At least Theo De Raadt and RMS are telling people they're utter fools for using a VM for security. Nobody else is, although if you ask any developer they'll immediately tell you he's right from what I've been researching around IRC on dev channels. Sure these guys are abrasive complete neckbeard assholes, but it's their near religious conviction to freedom that makes them stand out from the legions of corporate drones and technocrats that typically infest the field of security.

First of all things are not as cut and dry as you make them seem. There are numerous people who are widely recognized as security experts who advocate the use of virtualization based isolation. I may have fucked up by advocating for full hardware virtualization over / in addition to other sorts though, I need to do a little more research on this before I come to my final conclusion. Anyway, the three primary 'philosophies' of computer security are correctness, isolation and randomization. OpenBSD devs are religiously commited to security via correctness (although they do have very sophisticated randomization systems and have before pretty much anyone else, and they do have two tools for isolation but none of them use virtual machines or MACs) Others prefer security via isolation, knowing that only formally verified software is perfectly correct and that very very very little software has been formally verified, and that randomization can't protect from everything. It is a lie to say that any developer will immediately tell you Theo is right about this, many will immediately tell you that he is wrong actually. Most of the people who are big on security via isolation will tell you that he is wrong. Ask the Qubes devs or the FreeBSD devs or the Open Solaris devs or the SEL4 devs

I think Theo is right, the large corporate mass of security consultants are mainly con men who expertly fool people into thinking running 3 O/S's in a Virtual Machine is more secure than actually setting up 3 servers like you're supposed to so they can bid lower on contracts and keep most of the money for themselves. This is why Antisec, Anon, LulzSec, Carders and spammers have so much success is because of these con men fooling everybody into drinking snake oil.

I will concede that using full hardware virtualization MAY not be the best solution, it may be better to use other sorts of virtualization which will avoid all of the potential (although unsubstantiated and unexplained in technical detail) problems that you/Theo have brought up (well, IOMMU is also needed to fix some of them). I need to do more research before I conclusively say that full hardware virtualization should be abandoned in favor of using other sorts exclusively.

I personally am not convinced that you know what you are talking about and are not cleverly cherry picking opinions of certain people (none of which have yet said anything about anything other than full hardware virtualization, and I am not even sure I agree with his opinion on this yet I need to hear technical details and I need to talk with other security professionals including people who are fans of security via isolation) in an attempt to scare people away from more secure solutions. For one I don't think anyone other than maybe the OpenBSD devs would suggest using chroot for isolation over using jails for example. I will concede that you / Theo may be correct about the risks of full hardware virtualization though, I just need more time to research the matter. I also already know that the potential risks will not apply nearly as much if at all to other virtualized isolation solutions as they will to full hardware virtualization solutions.

Also I have heard from one hacker who I respect that breaking out of full hardware virtualization is actually more difficult than breaking out of other types of virtualization...I will ask him for his opinion on this issue as well. Essentially I need to do more research on this before I can make a more educated and conclusive reply to Theos comments on full hardware virtualization, but in short I am not sold on what he says and even if he turns out to be correct what he is talking about will apply much more strongly to full hardware virtualization than to other sorts.

edit: fixed technical error in what I wrote calling some systems that are not paravirtualization paravirtualization. Again, i am not an expert on virtualization and wish I had a few days worth of free time to read up on it in depth before I made a reply to this thread.

While x86 hardware has the same page-protection hardware that an IBM
390 architecture machine has, modern PC machines are a mess.  They are
architecturally so dirty, that parts of the video, keyboard, and other
IO devices are interfaced with even to do simple things like context
switching processes and handling interrupts. 

Has Theo never heard of IOMMU? From Wikipedia:

An input/output memory management unit (IOMMU) enables guest virtual machines to directly use peripheral devices, such as Ethernet, accelerated graphics cards, and hard-drive controllers, through DMA and interrupt remapping. This is sometimes called PCI passthrough.[30] Both AMD and Intel have released specifications:

    * AMD's I/O Virtualization Technology, "AMD-Vi", originally called "IOMMU".[31]
    * Intel's "Virtualization Technology for Directed I/O" (VT-d).[32] Included in most but not all Nahalem based processors. [33]

I know Qubes uses IOMMU and that it is supported by Xen, not sure if it works with virtual box or full hardware virtualization though. I think this should offer isolation on the layer that Theo is currently bitching about. Also lack of IOMMU with SEL4 is one reason the Qubes dev gave with why they went with Xen instead of SEL4, even though SEL4 is provably correct at what it does.

This is really fairly cutting edge computer science / computer security shit and I can't claim to be an expert on these matters, but it is the stuff I am currently researching and trying to understand. In the past few months I have come to one conclusion though and it is that I was wrong to suggest Virtual Box be used for isolation, I should have suggested Xen and I will personally be switching from Virtualbox to Xen. Last I checked Xen is much more difficult to configure and use than Virtualbox is, for the average user. If this is still the case, if you decide to go with virtualbox or nothing, I would still suggest virtualbox and isolating network facing applications away from Tor with it. The feds are not the caliber of attacker that Theo is worried about when he discusses security issues, and I am not convinced of his apparent claims that using virtualization is not only worthless but also a huge security threat in and of itself (I would really love for him to explain this in deep technical detail).

Also keep in mind when you talk with security researchers like Theo and others, that they are quite probably imaging the adversary as being on the level of an intelligence agency. My opinion is that just because the NSA can pwn your isolation layers doesn't mean you shouldn't use them when they will prevent the feds from pwning your ass.

I will make a more detailed response to this soon, but for now just let me say that it is pointless to argue with the OpenBSD devs because they are religiously devoted to their security philosophy, but just realize that their ideas are not shared by at least some number of other security researchers who are just as validly called leading experts. For example, I would love to see what the Qubes team has to say in response to this. I like to use OpenBSD for the guest because it has 64 bit ASLR, nx bit, and a bare bones highly correct base install.

You are absolutely deluded, if not stupid, if you think that a worldwide collection of software engineers who can't write operating systems or applications without security holes, can then turn around and suddenly write virtualization layers without security holes.

He is absolutely deluded to think that you shouldn't add layers for an attacker to go through, that they otherwise wouldn't have to go through, simply because they may be able to get through them. Of course most hypervisors are not perfectly correct (other than maybe SEL4 if it counts as a hypervisor), but that doesn't change the fact that they add an additional layer for your adversary to penetrate.

Is Theo aware that this discussion is in regards to hiding your IP address from an attacker? He really should be aware of the wider topic of discussion before he comments on the technique.

Anyway the argument I am seeing here is "virtualization isolation is worthless because virtualization technology is not perfectly correct, and only correctness matters". This is such a shitty way to look at things. OpenBSD hasn't been formally verified so you shouldn't use it because it may have security vulnerabilities that an attacker can exploit if they spend enough time looking for them. I wonder what he has to say in reply to that.

if the actual hardware let us do more isolation than we do today, we would actually do it in our operating system. The problem is the hardware DOES NOT actually give us more isolation abilities, therefore the VM does not actually do anything what the say they do

First of all this sentence makes little sense to me so it is hard to reply to it. I know little about hardware isolation so I can not really comment on this yet I need to read some papers on it that I have but have not had the time to read. I am thinking of things like the Tor routers that run Tor on a purpose specific device rather than requiring the user to run it on their operating system, however I think he is probably talking about things like AMD-v. Please ask him to clarify this sentence and go into technical detail.

While x86 hardware has the same page-protection hardware that an IBM
390 architecture machine has, modern PC machines are a mess.  They are
architecturally so dirty, that parts of the video, keyboard, and other
IO devices are interfaced with even to do simple things like context
switching processes and handling interrupts.  Those of us who have
experience with the gory bits of the x86 architecture can clearly say
that we know what would be involved in virtualizing it, and if it was
so simple, we would not still be fixing bugs in the exact same area in
our operating system going on 12 years.

I need more time to think on a response to this than I have right now, but I would like to mention that I am not even talking about virtualizing the x86 architecture but rather x86-64, although what he says may very well still hold true for that. Also now is a good time to mention the distinction between virtualization solutions like virtual box (full hardware virtualization) and systems like Xen (paravirtualization). I think that paravirtualization will avoid all of the potential problems Theo mentions here, and this now further strengthens my belief that Xen should be used over Virtualbox or VMware.

Also allow me to say that I am not an expert on virtualization or even on computers especially as compared to people like Theo, but I can recognize that as good at security as he is other people who are just as good as him or possibly even better (I can not tell) use different strategies than he does.

We know what a VM operating system has to do to deal with the PC
architecture.  It is too complex to get perfectly right.

Well I am not an expert on this matter, but I think sel4 offers a provably correct layer of isolation in addition to using paravirtualization. I need to research this more before I make any definitive claims though.

And now you've entered into the layered approach where *any error* in
the PC model exposed to the client operating system is not just a
crashing bug -- it is now exploitable.

I don't follow his logic of how everything is now an exploitable bug please ask him to clarify for me. Layered approach aka defense in depth is favored by many security professionals. And again please inform him that this discussion is related to a technique for hiding your IP address from an attacker who roots your firefox VM, while using Tor in another VM and host only routing with firewall rules on the host.

It might be nice, but it is stupid.  And anyone who thinks there is
any security advantage at any level knows nothing about PC
architecture..

That is a pretty big claim to make considering there are a ton of world leading security research teams who are focusing on using virtualization in this way for security advantages. Then again, Theo is a world leading security expert. He knows more than I do about computer security. So do people who disagree with him.

This massive move towards VM use is a worrying trend and I am scared
of the side effects we will face from so many people (essentially)
choosing to run 3 operating systems instead of 1 ... and doing this
when their guest choice is 'OpenBSD for security'. I really wonder
how people arrive at such a position... without logic or technological
understanding, I suppose.

He thinks it is a worrying trend. The people doing it think it is the future of computer security. I see Theos logic as being that virtualization being used for isolation like this is a bad idea because the virtualization technology is not perfectly correct and can be broken out of. My response to this sort of logic is that people should work on making a correct hypervisor then. Or that you shouldn't use OpenBSD because it isn't formally verified, sarcastically of course.

I will make a better more detailed response to this post soon I don't have time right now to spend the time required to think and research on a better post. Arguing with people of this caliber is a good way to look like a dumbass because they are widely recognized as leading experts, but I know that I have logic in my arguments and that others who are equally as impressive as Theo would also argue against him in similar ways as to how I have. However, I wish he was having this debate with the Qubes research team or the SEL4 people instead of me.

You can also go total neckbeard commando text only and just dual boot OpenBSD 5.0 + something else (Debian? Win7?).
Your encrypted OpenBSD 5.0 partition does nothing but drugs business, hacking, sending threats to the NSA ect. You don't need Xorg you can do everything command line like encrypting/decrypting emails and sending them through Tor from your @tormail account. Type 'mail' in the command line and get used to working with it (now you have zero email exploits). Then you have persistence, and aren't constantly grabbing new guard nodes which is a slight anonymity flaw of using Live CDs like Tails/Liberte. For internet use lynx in your business only O/S. This forum doesn't look as nice using lynx but still works. If the NSA ever (unlikely/tinfoil hat) compromised it and hid 0day firefox, IE and Aurora exploits it wouldn't effect you in your bomb shelter operating system from the past nobody can break.

Since BSD is transparent you can cron job to nuke absolutely all evidence (history, emails, whatever) there's no bullshit MS page file or system restore backups saving critical information. Then reboot back into your Win7 machine to look at porns, play games, whatever you want.

This is a good softraid tutorial on setting it up
http://geekyschmidt.com/2011/01/19/configuring-openbsd-softraid-fo-encryption

Then of course encrypt the entire HDD for fun an profit with truecrypt, and use truecrypt containers for crazy sensitive stuff like ordering info or use LUKS, or keep it on a separate USB encrypted key

Not having persistent entry guards is actually a major anonymity flaw there is nothing slight about it. I think liberte actually does have persistent entry guards but amnesia doesn't.

Theo De Raadt is one security researcher and his views on virtualization being used for isolation / security are not held by all security professionals, for example the team that made Qubes would obviously disagree with him. My favorite thing to say to people who are rabidly against virtual isolation is that if correctness is the only way to be secure people should make a correct hypervisor because then the attacker wont be able to break out of it . Also OpenBSD is big on correctness via constant code audits by professionals, but a significant number of security experts will be quick to point out that OpenBSD has pretty shitty support for security via isolation, it is kind of controversial that they have no mandatory access control systems and its support for virtualization technology is shitty as well. Don't discount the real security advantages of virtualization being used for isolation based on the words of one person, even if he is a world leading security expert, because others of the same caliber disagree with his position.. Defense in depth is imo the best strategy and it is a shame that the openBSD team thinks they can ignore isolation in favor of code correctness, OpenBSD may have very few remote code executions and a history of high correctness but when you run OpenBSD you will be running other non-audited applications that have plenty of zero day vulnerabilities. It would be nice if you could isolate them with mandatory access control systems and virtual machines, but good luck doing that on OpenBSD.