When I was experimenting with the memorist, Sarah's Red Magic didn't include Summon until she reached Level 1 in the Summoner job, thus adding it to her list of chosen abilities. So if the ideal memorist were to gain access to all 3 forms of higher-level magic, there would be 6 abilities for the 4 slots, not 5 minimum of abilities that would be gained from mastering the Red Mage job: Red Magic, Attune Blade, Magic Blade, Doublecast, Memorandum, and then add Summon.
Ultimately, in my case, the Memorist job was a waste. I decided that for my 2nd playthrough I would have at least 1 character master every job, and I used Sol as the memorist, but he had dabbled extensively in every job possible because changing from one job to another and getting F-Abilities is what makes the game so fun. In that same playthrough, Sol mastered the Red Mage, Black Mage, and Seer jobs. It goes without saying that on the postgame superbosses Sol was a Level 20 Seer with doublecast and magic font in his ability spots.