Like many of you mentioned it's probably my ISP, it loads fine with VPN but without VPN takes hell lot of time so I am using VPN at the moment and it works butter smooth so it concludes there is nothing wrong from the forum side and if anyone having the same issue as me just do the same, use VPN or TOR.
I want to keep this thread open because I'm curious why only altcoinstalk because I never had any problem with bitcointalk, so that rules out my ISP is restricting crypto sites.