Each of those "current" directories was a symlink to the version I was using. Then, I put bin directories in PATH variable.
This all worked fine for me. But I learned that ubuntu was recommending user-specific environment variables in ~/.pam_environment. See https://help.ubuntu.com/community/EnvironmentVariables
I decided I'd move my configurations to the proper place and improve my linux knowledge. Being careful, however, I've only commented out the sections in /etc/profile so I could quickly revert them back, in case it somehow didn't work as expected.
On ubuntu help page mentioned above, it says:
~/.pam_environment - This file is specifically meant for setting a user's environment. It is not a script file, but rather consists of assignment expressions, one per line.
Assignment expressions. Ok. I'll just remove those "export" keywords.
This should do it!
I rebooted the machine, got to the login screen, typed in my password, and then, typed in my password, typed in my password, and then.. errr... what is going on here?
It appeared to me that I wasn't able to login to my own account anymore! First, I was just annoyed. And then, I was afraid. How am I going to fix this, since I only have my own login to this machine, and the guest account can't do anything useful like changing someone else's configuration files.
Luckily, I was still able to use the browser logging in as "guest".
After googling around, I found I could boot into recovery mode by pressing left shift key while booting up. And then, googling around a bit more, I also discovered that the .pam_environment file has a special configuration format different to the usual /etc/profile.
What I had to follow was to use pam's configuration file style.
The whole thing was not quite clear to me, but reading man pages of pam and pam_env.conf gave me a hint as to how I should write my .pam_environment file.
So, after many reboots in recovery mode, dropping into command line as root, editing my .pam_environment to try different things, I came up with a working configuration:
I hope ubuntu help page will describe it a bit better in the future.