Creating a new Map Window can be done from a context menu in the Project Explorer. Right-click on the Session node to contain the new Map Window and select 'Add Map Window...' which will open an input dialog.
Type in a unique the name for the Map Window and select OK.
Map Window names within a Session must be unique. All the settings used by a Map Window will be inherited from its Session until overridden.