On my home PC, I have a 'normal' copy of Firefox, then I install Portable Firefox on every other machine I need to sync. I then zip up my profile directory (see Daniel's comment above for location) and upload it to a free online file hosting service like DivShare or put it on a USB flash drive or similar. I then copy this profile into the Portable Firefox profiles directory on every other machine.

This seems like a pain to do on a frequent basis, but by using the Foxmarks and Google Browser Sync it automatically syncs bookmarks, history, cookies, saved passwords and even saved tabs/windows without having to replace the profile. So, in actual practice, I find I only have to replace the Portable Firefox profile when I make major changes to my main Firefox, such as new extensions or changing browser settings. Hope this helps!
I also use Firefox Portable on a USB stick (as well as OpenOffice). I've got a really cool setup using U3 LaunchPad and a free auto-loader package called Xtraveller (http://software.u3.com/ -- Find it under the utilities menu - if your USB stick supports U3 apps). I have set up LaunchPad to auto load Xtraveller which in turn auto-loads PortableApps and then Firefox Portable. All of this is takes place without a single keystroke from me.

With the addition of Keepass Portable (a password manager) which can be configured to auto-load if desired, this process pretty much ensures that my web-browsing experience is the same no matter whose computer I am on. Additionally I don't have to install any software or leave anything unwanted in someone else's cache (passwords, browsing history, etc.)
if you want to access the same profile with multiple users/multiple computers on a LAN, you can edit the profiles.ini file (located in %windir%\Documents and Settings\%user%\Application Data\Mozilla\Firefox), changing the IsRelative value to 0 and typing the path you want the profile to be stored at, e.g. C:\Firefox Profiles. Then just copy your existing profiles to that folder, and edit the profiles.ini file for all users/computers to point at that location.