I been to Japan 5 times for vacation, 3 times with the missus. We feel we've only scratched the surface. This is partly due to the fact that we only have basic Japanese language skills. But as others have mentioned, there is enough English signage to get around. And I suspect the signage will be enhanced more so in preparation for the Olympics. There are going to be times where language is going to be a problem (eg. Pre-ordering and paying for food at a machine) but we've always had friendly help from locals. And the language struggles are part of the fun IMO.
We found reading through a guidebook also really helped up prep for the trip as it explained pragmatic stuff like how to use the public bus.
We've done a lot of the museums and stuff during our initial trips. Glad I did it, don't have to do it again. Our recent and future trips focus on food and culture. Bonus if we line up with a festival/matsuri.
I agree, the Imperial Palace tour in Tokyo is pretty disappointing. In Kyoto, the gardens are a bit nicer.
If you like Japanese food, it's a foodies paradise; even "value" places. The basement food floors of the nicer department stores are amazing with so many different food types in one place. Food arcades like Nishiki Market are also interesting.
Japan is doable DIY but it does require a chunk of planning, particularly your first trip. Depending on how long you go for, the typical places to visit are Tokyo and Kyoto IMO; ideally with an open jaw flight. From there, you can easily add day trips outside city core, particularly if you have a train pass. The main JR pass is typically only worth it if you plan to travel from Tokyo to Kyoto/Osaka and return. You can also buy regional passes at a discount overesea.
Japan does not have to be expensive. But it obviously depends on your lodging and meal choices.
I highly recommend getting a data sim.
Google maps is surprising decent but will struggle finding places that don't necessarily have an English name. One way around this is to research ahead of time and pin locations of interest on the map associated with your google account.
Some cities have a free volunteer tour guide program like as follows:
TOKYO FREE GUIDE - Volunteer Tour Guide Service in Tokyo
We've found the following sites helpful for research:
japan-guide.com - Japan Travel and Living Guide
Wikitravel - The Free Travel Guide
Japan Guide is kind of nice as it give sites ratings as well as a summary.
The Hyperdia app is great for train schedules and routing
HyperDia | Timetable and Route Search in Japan.