The port is apparently consists of a six-dot brass contact, resting inside the groove meant for the bottom strap connection if holding the Apple Watch with the digital crown sitting on the right-hand side. Apple is apparently using it for diagnostic purposes. According to Techcrunch:

The reason I ask is that the Apple Watch has a port that the company has yet to show off. It's being used for diagnostics and direct access to the Watch operating system, but it's feasible that could be used to connect accessories in the future.

Regardless of whether or not connected accessories are on the table for the Apple Watch at this point, it seems likely that this port will be the access point for Apple if you need to take your Watch in for service. It's probable that it could also be used for factory loading of software, similar to the USB port on the back of the current Apple TV.

It's possible that we could learn more about the port during Apple's "Spring Forward" event, which will take place this coming Monday, March 9.

Update: The folks at 9to5Mac say that their sources indicate that the port is a Lightning port used for testing, and will not be included on the final consumer product.

Sources: TechCrunch, 9to5Mac