Humor is an excellent way to ease tension in the workplace. It can serve as a way to make people feel comfortable and even more connected to each other.
Remember that different people have different senses of humor. Before making a joke, try to make sure that it won't be construed as offensive.
You can try making a lighthearted joke about something situational. For example, if it's a nasty, rainy day, you can jokingly say, "Lovely day, isn't it?" It may not be the most original comment, but things like that can lighten the mood.