Lazarus is very lightweight IDE in the end, at least compared to Delphi. What "too much" is there that you wouldn't use? It is designed to fit also in purpose of making just console applications. There is just the component palette, which you can almost completely hide by making the palette window just show other menus. You could possibly even compile Lazarus so that none of them are showing, because they are "installed components" like any custom ones.

It's possible to code pascal with just Notepad, but you would miss features like code completion, debugging, fpc compiler settings and so on.

Another thing that possibly makes Lazarus seem heavier than it is, is the splash screen. Start Lazarus with parameter:
lazarus.exe --no-splash-screen
... and you will see that Lazarus starts up in a fraction of second.