How can I get keyboard shortcuts like TAB to work on a modeless dialog on an MFC control?

Here is what you need to do to get the modeless dialogs working properly. Basically all you need to do is send private message from the dialog on activation (WM_ACTIVATE and nState:WA_CLICKACTIVE) and at that point Set the focus to the OCX and make the dialog the active window. Following is the detailed description of how to do it:

First define a custom message using #define or preferably RegisterWindowMessage().

Then, create a member variable of type pointer to the dialog class you need to create in the OCX class.

Wherever you want to create the dialog, Call SetFocus() to get the focus to the control and then create the dialog using new and create.

void CModelessmfcCtrl::CreateModeless()
{
// TODO: Add your dispatch handler code here
	SetFocus ();
	myDlg = new CMyDlg (this);
	myDlg->Create (IDD_DIALOG1,this);
}

void CModelessmfcCtrl::RemoveModeless()
{
	myDlg->DestroyWindow ();
	delete myDlg;
}

In the Dialog's Class (CMyDlg), handle WM_ACTIVATE (OnActivate) to send  the private message to the control.

BOOL CModelessmfcCtrl::PreTranslateMessage(MSG* pMsg)
{
	if (::IsWindow (myDlg->m_hWnd))
	{
		if (::IsDialogMessage (myDlg->m_hWnd,pMsg)) return TRUE;
	}
	return COleControl::PreTranslateMessage(pMsg);
}

handle WM_MYSETFOCUS in the controls CPP to set the focus and set the active window.

LONG CModelessmfcCtrl::OnMySetFocus (WPARAM wParam,LPARAM lParam)
{
	SetFocus ();
	myDlg->SetActiveWindow ();
	return 0;
}

Add this in your message map:

ON_MESSAGE (WM_MYSETFOCUS,OnMySetFocus)

And Add this to the OCX class header (*ctl.h):

This should make the keyboard to work properly on your modeless dialog.