So does that mean better to have them without?
Depends on how the link will be accessed.
Let's say you have a file named "smiley.gif" and your site URL is wakawaka.com. On your server you have a single folder named images off of your root folder so the full url to the images is wakawaka.com/images/smiley.gif.
Now let's say you have two pages, one named "page1.html" and is in the root of your web site; the url looks like wakawaka.com/page1.html. The second page is named "page2.html" and it resides in a folder named "docs" off of your root folder; the url looks like wakawaka.com/docs/page2.html.
Now both page1.html and page2.html is supposed to display smiley.gif when shown to the user.
If you use "/images/smiley.gif" then the image will be shown as expected on both pages.
If you use "images/smiley.gif" then it will only be shown on page1 and not on page2 because the URL is interpreted as "wakawaka.com/docs/images/smiley.gif" which doesn't exist.
With XF everything is served relative to the folder that XF is installed in so that is why right now both links are working for you. If the only thing on your URL is XF and all of your pages are being presented from within XF and all of the content is rendered using XF (eg: a smiley code) then you don't have much to worry about. If, on the other hand, you might have several things installed at your URL and, for example, you have /images/smiley.gif on the server and XF is installed physically in folder named "myforum" (eg: wakawaka.com/myform/index.php") you'll need to use the "/' in front of the relative URL.
Cliff Notes (or "tl;dr" for the younger generations): You usually can't go wrong with using the "/" in front.