No, you would not ONLY count your dividend-paying stocks. You would count ALL the stocks you plan on using in the future, whether by dividend-generating OR by selling them.
And no, you would not ONLY count real estate you are renting. You would count ALL real estate that you plan on using in the future, whether by renting it OR by selling it.
If you buy an asset that will not generate income in the future, like a pencil that you plan on sharpening to it's useful end, it is an asset that should show on your personal balance sheet, but it is NOT an investment that would qualify as part of your portfolio to be used for retirement because it will never generate any income for you.
Similarly, if you plan on living in your home until you die without selling or renting it, it should be part of your balance sheet, but it should not be part of your portfolio from which you will be applying your 4% SWR.