The issue below was the initial issue, which has somehow resolved itself, but now I have another issue that resulted from that, and NOW my FPS die when 20+ players/mobs are nearby.
My chunks load fine in singleplayer, and my fps are fine there too. For some reason, when I join a server, my chunks load extremely slowly and my fps drop from where they normally are 120, to the 30, sometimes even in the single digits, fps. Here is a video showing my issue.
Also I know I am using a modified client, but I tried this with vanilla versions and the same thing happens.
My brothers both get 100% fine and working fps to the server. The server is fine. Recently, I have noticed it actually isnt the server's fault, however it is the fault of having too many mobs AND players within render distance. I changed the title to this thread. So basically those 30 or so mobs and players in the lobby affect my fps. However, I don't know why these mobs would affect my fps when, in the past, they were FINE on my computer (got a solid 120 fps consistently) now my fps almost instantly drop when I am in range of over 20 players or mobs.
He is saying that you don't have a dedicated graphics card and shouldn't expect to get a lot of fps. Everything you describe can be referred back to that weak graphics chip you got.
I disagree with that statement. My fps used to be fine and in single player worlds my fps are fine so long as I don't spawn a lot of mobs (this is not really an issue though) and my fps are even fine in CS:GO, where they rest at a consistent 80.