For ten years my sole car was a 1984 Honda CRX 1.5 that I bought new. While not quite as small as the Toyota 800 at three inches longer but marginally lighter than the Smart, it was about as tiny as production US-market cars got in the 1980s. I absolutely loved it. Kart-like handling, surprisingly roomy for two, great economy, decent luggage space, and non-stop fun even on dull errands. I never felt seriously threatened by its size relative to other traffic.

While it may be small in comparison to today's small cars that is because today's cars are too damn big. You really don't need all that size just to get around in. I would actually love to have a car that small to drive on a daily basis as my Geo Metro just isn't as maneuverable as I would like ( It can't turn around in the width of a residential street without backing up). I think my next car will be an old school Mini or Fiat 500. After all, once you start driving small cars it gets addictive and makes you yearn for something even smaller!
As far as safety goes, it's all relative. While I know that an accident with an F150 in such a small would kill me instantly, I also view it as being safer than a moped or a motorcycle. Plus, I get a roof over my head in a rain storm and see fuel economy numbers equal to or better than those of co-workers who ride motorcycles, so there is an advantage to a tiny car over something on 2 wheels.

On the one hand, yes obviously the smart is a lot safer in a crash. On the other, I bet the occurrence of crashes in the remaining Toyota 800 sports is nil. There's a reason classic car insurance is so cheap: nobody ever crashes them.