Board Thread:Lords of Shadow 2/@comment-216.170.54.18-20140301115207/@comment-71.211.86.196-20140315200937

if you look at the Notes on the page for Carmilla's lair it states this.

'Like all other places of the castle, it is a physical manifestation of his memories and wishes, and past events (it exists as its own sort of time travel/pocket universe). The castle represents the period shortly before the destruction of his castle by the Great Explosion when it was under siege by the forces Roland de Ronceval. This realm overlaps Castlevania City, and there are places where the two realities merge. Throughout his quest, Gabriel physically disappears to the realm (and is hidden from Zobek), only to be teleported elsewhere in the city when he returns.'

so you would be right as the Castle being in a pocket dimension. this may be alluding to there being more to the Forgotton One's power than we know, is it used a majority of it to rip a hole between realms to travel through ex; the dimensional prison->The Underworld->The mortal realm/earth).