I'm regretting not having played Parasite Eve in a long time now. I can't remember exactly what Junk is used for, but I don't think you'll screw yourself by getting rid of a piece here or there. I know there's a special powerful weapon you can get if you collect 300, but it's not remotely necessary to do.

As for upgrades, I seem to recall that later in the game you can use old weapons to improve the stats of new ones, so you don't completely lose upgrades by using them on less powerful weapons early on. I don't remember having any real problems when I first played through the game many years ago, and I'm sure I was not playing as efficiently as I could at the time.

Speaking of Parasite Eve though, I was having the urge to play it again soon. Such an awesome game, I might have to start another playthrough soon.