You can go in. If it is a classroom, there will be a notice on the blackboard or on the door. If no one is in class, just go in; If someone comes to class, leave and change to a new classroom.
This university is open and free. Relax, this is not a forbidden area. Good luck!