Japan has by far the best public transport system in the world. Major cities are linked by 200 mph trains, every large town has a frequent rail service, and even the tiniest town has several buses a day to the nearest station. Every city has a dense and frequent bus service, while in Tokyo you're never more than a five minute walk from the train or subway. Japanese trains are a delight to ride, with constantly changing views of the towns and countryside. With the exception of the Tokaido Shinkansen between Tokyo and Osaka, they are rarely anywhere near full.
So given all that, why would you ever want to drive? The traffic in towns is difficult, parking is expensive and hard to find, while navigation, if you don't read at least some Japanese, is bound to be hard.
But... if you want to see the countryside, to stop at leisure to admire the view, to wander at will - you need a car as much in Japan as you do anywhere else. It's possible to view rural Japan from a bus, I've done it a couple of times. It works, but it is very constraining. When we visited Wakkanai in the far north of Hokkaido, we couldn't rent a car (I'd forgotten my international driving license, which is essential). We did manage to get to Cape Soya, Japan's northernmost point, using the bus, but we had a very tight timetable and didn't have time to see all the various monuments that have been established there.
Practicalities
Car ownership is relatively uncommon in Japan. You can't legally own a car unless you have off-road parking for it. That's why there are so many really tiny cars - often you will see them squeezed into a garage so small that you wonder how the driver gets in and out.
As a consequence, car rental is ubiquitous. Close to the main station in every town, you'll find Toyota Rent-a-Car, Nissan, Nippon, Times, and a couple of others. It's taken for granted that people will use the train for the long journey, and a car only for the very final part.
But you didn't see any of the familiar names there: Avis, Hertz, Enterprise... You can make reservations through them, but they will be fulfilled by Japanese companies with different names. For example, I recently used Expedia, which made me a reservation through Europcar - but I actually had to go to Times Car Rental, and nothing in the reservation told me that, nor gave me their correct address.
It's better to book directly with a Japanese company. If you have a few days in Japan before you need the car, your hotel concierge desk will generally be able to do this. Some companies have english-language sites (e.g. http://www.timescar-rental.com), and there is a site called ToCoo! you can use.
The Japanese drive small cars. This is a good idea, because the roads are narrow, city streets sometimes frighteningly so, and parking spaces are small too. A "mid-size" car in Japan would be one of the smallest cars on the road in the US. Don't try to get a normal-sized car by US standards, it will cost a fortune and you won't be able to park it.
You can not rely on your own driving license. Japan is a country which absolutely requires an International Drivers Permit (IDP). If you don't have one, when you show up they will apologise a lot but they won't give you a car. In most countries these are available through motorist organisations such as AAA (US) and RAC (UK). For some countries, including France, the procedure is different, and you need an official translation of your license instead.
The pick-up procedure is generally painless. You find the office (usually the hardest part), you present your passport and IDP, and they show you to a car. Make a careful note of where you are, because you will have to find it again when you return the car. Drop-off is also painless, apart from finding the place, and assuming you haven't damaged the car. Most places have little experience renting to foreigners, and they are very relieved to see you back again. I once dropped off a car a few minutes before closing in a small town, and the staff were standing at the roadside waiting for me!
If possible, get a map from somewhere (for example, your hotel concierge) showing the exact location of the rental office. Otherwise, you may well spend a miserable half hour wandering up and down the vicinity of the station looking for it. Also, be careful which side of the station it is. Most stations have entrances on either side of the tracks. Often rental places are on the "wrong side of the tracks", away from the main entrance.
If possible, get a map from somewhere (for example, your hotel concierge) showing the exact location of the rental office. Otherwise, you may well spend a miserable half hour wandering up and down the vicinity of the station looking for it. Also, be careful which side of the station it is. Most stations have entrances on either side of the tracks. Often rental places are on the "wrong side of the tracks", away from the main entrance.
Driving
Driving in Japan is easy. Speed limits are low - 50 km/h (30 mph) on most roads, only 80 km/h on highways. People sort-of obey them, as elsewhere - most traffic is doing closer to 60 than 50, but not more. Important signs are easy to understand. There are lots of signs that are only in Japanese, but they're not giving directions. They say things like "sharp curve 200m ahead" - useful but not indispensible.
Roads in Japan can be strange. It's the only country where two-lane divided highways are common. That's right, just one lane in each direction, with no possibility of overtaking. It matters less than you'd think, because the speed limits are so low and everyone is driving at the limit anyway.
The road around the western coast of Shikoku is sometimes barely two lanes wide, then for a few km it's a four-lane highway. It alternates like this for a long way. The giant refrigerated trucks carrying fish to market don't care, they just barrel along at 60 km/h whatever the state of the road.
In Hokkaido, we were following a road along the coast that got narrower and narrower, and eventually turned into a dirt track. We were just thinking about turning round, when in the middle of nowhere it turned into a four-lane highway - better than the main coast road running inland. There was no apparent reason, no factory or houses. I'm sure there's a story behind it.
The hardest part of your journey is likely to be getting out of, and later back into, the town where you rent the car. Signposting in towns is almost non-existent and roads are often not quite as they seem on the map. You should really prepare carefully for this. Print bilingual maps from Google, possibly at several different scales. Even if you don't know the exact location of the rental office, it will be somewhere close to the main train station.
The hardest part of your journey is likely to be getting out of, and later back into, the town where you rent the car. Signposting in towns is almost non-existent and roads are often not quite as they seem on the map. You should really prepare carefully for this. Print bilingual maps from Google, possibly at several different scales. Even if you don't know the exact location of the rental office, it will be somewhere close to the main train station.
It goes without saying that you should drive carefully and be sure to stay legal. Even if at home you sometimes interpret the rules a bit flexibly, don't do it in Japan. It's said that if ever there is an accident involving a non-Japanese driver, they will always be at fault. I hope I never find out whether it's true - I really don't want to spend a few nights in a Japanese jail. And nor do you.
And of course don't even think about drinking and driving. There's nothing to do and nowhere to go in the evening in rural Japan anyway.
Fuel
You will have to refuel the car at least once, just before you return it, and maybe more if you drive a long way. Fortunately, this is easy. Gas stations ('gasoreen stando') are ubiquitous. In towns they are often hidden in a space under a large building, with only the sign visible from any distance. The most common brand is Eneos, with a distinctive orange sign. Japanese for "fill 'er up" is 'mantan', but gestures work fine too. Most gas stations are fully staffed, you just sit in your car while eager young men run around filling it and cleaning the windshield.
Sometimes there are surprises. We once refuelled in the depths of rural Hokkaido, and were rewarded with not just one but half a dozen large boxes of Kleenex. In our tiny car they took up way too much room, but it seemed churlish to throw them away.
Navigation
Now we get to the hard part. Japan has really excellent maps. Mapple publish road atlases, one for each prefecture, at a scale of 30,000:1, with enlargements for towns. There's also a very useful series of folding maps at 120,000:1 (about 2 miles to the inch). These are all available for the whole country at big bookshops. You can often find the local ones even in convenience stores. But... they are only in Japanese.
The best bilingual map of Japan is "Japan: a bilingual atlas", at a scale of 260,000:1. That's enough to show all towns and significant roads, and it has enlargements for major towns. Used in conjunction with one of the bigger maps as a sort of translation guide, it will get you there.
Google maps is also bilingual. If you have a tablet and a reasonable deal on roaming, you can use that. Even if not, you can use it to prepare your trip and maybe print some of the more critical bits and take them with you.
Directional signs on roads are written in English characters as well as Japanese. As everywhere, they won't necessarily tell you about the place you want to go to, so you need to have an idea of the route and other places that might be in the right direction.
Every rental car has GPS, but not only are the maps all in Japanese, so are all the controls. I do read Japanese, but I've still never managed to get anything very useful out of them. On one occasion the GPS had been programmed to go back home. At every junction and turning, a cute Japanese girl voice would plead with me to make a U-turn. It took me about an hour to find the right place to press to shut it up.
The GPS can be useful in towns. While I really strongly recommend against driving in towns, at some point you'll have to get to and from the rental location. Even though you can't read the map, you can get a general orientation get yourself headed in approximately the right direction. These days, they show gas stations too, which can be handy (look for the tiny Eneos logo).
Conclusion
If you really want to explore the Japanese countryside, a rental car is the only way to do it. It's a bit daunting, but not too hard.
Here is another useful link with some more information: http://www.japan-guide.com/e/e2022.html
Here is another useful link with some more information: http://www.japan-guide.com/e/e2022.html