Where to Buy Japanese Food
A Japanese grocery is your best bet, obviously. Lacking that, any Asian grocery is likely to have at least a few Japanese items, or a Chinese/Korean/Vietnamese substitute. Japanese food is slightly more expensive than food from the rest of Asia, so the higher-class the grocery, the more likely they are to have what you're looking for.
Ordinary American groceries, especially large supermarkets, are increasingly likely to stock Asian food. You can definitely find staples like soy sauce and rice vinegar there, sometimes not even shelved in the Asian section. If the store has any pretense to internationalism at all, you can also buy rice, nori, and curry powder there. Prices are high, though. My local Stop & Stop asks the same price for a 6-serving box of curry that the Asian groceries ask for a 12-serving box. The selection is also low, and is usually confined to a few popular brands. If all you have is the local supermarket, you can do fairly well, but hit the Asian groceries when you can.
If you can't find something in a local store, the miracle of the Internet is here to save you:
Snacks
Groceries
