All of this is based on what I experience and doesn't necessarily describe the true nature of Japan/japanese people.
As most people have heard, Japan is supposed to be a country of service.
That famous country where the people will greet you with a smile and don't accept any tips.
And they are right.
When I was slightly lost about the procedures at Osaka Airport, the workers there tried to help me. Even when they weren't able to speak english and noticed that I couldn't speak much japanese, they tried to explain everything, they even walked with me through half the airport or made funny gestures to show what they meant.
When I was at a drug store there was even a clerk that apologized for looking at me without me wanting something. Or another worker that made a deep apologetic bow when he realized that I couldn't understand what he told me about their discount offer. And when I looked for a plug adapter to use the electricity of my apartement I even suddenly had four workers around me, trying to figure out, if they had such a thing and if it would work or if I would waste money (and they were only at a price of 200 Yen).
The only untrue thing is that everyone is all smiles.
Most clerks don't smile at you, if they don't feel like it. Some force a smile, some just seem to be in a good mood, but most simply look like you would look, if you would be sorting stuff into a shelf for 6 hours already. But they are still very friendly and helpful.
Also the not-workers are very nice.
I had to ask a woman for directions and she hadn't gotten any idea. So - despite me saying that I don't really understand a lot of japanese - she talked to me for two minutes, pointing towards the directions that she guessed had the shop I was looking for. And she apologized around twelve times for not being able to give me a exact answer.
Or when my landlord thought that I must be at least 30 and the teacher of my tutors. Once she realized that I was the same age as them, she was so embarrassed that she apologized again and again, bowed a lot. And even when I met her later the day she immedately started apologizing and bowing again.
Even my tutors, who behave pretty "international" apologize for almost anything. They didn't directly understand what I asked? They apologize. They said something in english and I had to correct what they said (or, more actually, I just ask, if they mean "supportive" when they say "saportafu")? They apologize. Even when we looked for a parking spot and I wondered where we would go to, they apologized for not informing me about that. And - of course - when I mentioned that they use some words I never heard in my japanese lessons before, they apologized.
Bowing, apologizing and being thankful is something that everyone here does and that people obviously need to adapt. So people that are very proud and like to make a point or to have the last word will have a hard time getting used to the fact that here isn't just one person wrong. If someone like my landlord thinks I'm a teacher and apologizes for this mistake, then I need to apologize for her making the mistake. And it doesn't matter that I never actually wanted to do that.
Thats just the way the onigiri rolls here in Japan.
There are two words that are often used as a apology: "Gomenasai" and "Sumimasen"
Gomenasai is the actual word for an apology, meaning "I'm sorry". Said after a mistake, being rude or simliar.
Sumimasen is more like "Excuse me" and used when you will do something that others might find offensive and rude (like needing to get through a group of people) or when you simply want to get someones attention.
It's not too bad to mix those two up (at least for as long as you are obviously not japanese), but the difference is still there.