Appendix A of the 4D v11 SQL Design Reference states that "The name of the help file is identical to the name of the database’s structure file." Normally that is going to be the same name as the .4DB or .4DC file.
However, if you are working with a Mac OS X package (.4dbase), the name of the help file must match the .4dbase name instead ot the structure file name. If the name of the .4dbase and the structure file differs, then the custom help file name (.htm) must match the .4dbase file. The help file will not be displayed in the Help menu if it's name matches the structure file.