I did some digging on the subject and this is what I found: The law seems to be different depending on whether the non-US spouse is a permanent resident who wishes to remain in the US or a non-resident.
If the spouse is a non-resident, non-citizen, then the estate tax exemption is very low, around $60K to $100K depending where you look. So, the non-resident, non-citizen spouse would receive only the first $60K-$100K tax free then would have to pay estate taxes on the rest. The non-resident, non-citizen spouse would then be able to dispose of the remaining assets as he/she wishes, including permanently moving those assets abroad.
If the spouse is a non-citizen but a resident (and wish to remain a resident), then the non citizen spouse cannot use the unlimited marital deduction but the estate will only be taxed if its value exceeds the unified credit amount ($3.5M for 2009, unlimited for 2010, $1M for 2011 and thereafter). For large estates, there are ways for the non-citizen, resident spouse to use the unlimited marital deduction. It involves using a type of trusts called QDOTS: upon the death of the US-citizen spouse, his/her estate passes to the trust entirely free of estate taxes. The non-citizen, resident spouse can enjoy the income from the trust (estate tax free), but the principal cannot be disposed of without first paying the proper estate tax. When the non-citizen, resident dies, estate taxes are paid on the assets inside the QDOT and the remaining assets are passed onto designated beneficiaries.
I have read in several places that there are further complications if the non-citizen, resident spouse subsequently decides to move abroad permanently with his/her assets, hence effectively becoming a non-citizen, non-resident. I cannot find much detail about these complications, but my feeling is that estate tax might be levied on the difference between the $60K-$100K exemption for the non-citizen, non resident, and the unified credit amount.
Things to keep in mind: 1) When a US citizen is married to a non-citizen (resident or not), the gross estate of the US citizen includes all jointly owned properties in their entirety (unless the non-citizen spouse can show that he/she contributed significantly to the joint property). Therefore estate taxes might be owed on all jointly owned properties as if they were entirely owned by the US citizen spouse. One way around that is to avoid owning assets in joint tenancy. 2) State estate taxes might have very low estate tax exemption for all non-citizen beneficiaries, whether they are residents or not. I have read that Washington state for example only has a $60K exemption.