Here's my attempt at the problem. This first post contains all of control.lua, the second post contains explanations of the changes:

----------------------------------------------------------------
-- SomeTrainStationsAreOptional v1.0.1 control.lua
----------------------------------------------------------------

-- TODO
--    on_train_schedule_changed ?
--    make some options controlled via mods settings (dbg, pat)

----------------------------------------------------------------

local dbg = false
local pat = '?'

----------------------------------------------------------------

local function myTrainChangedStateHandler(event)

    local idx = event.train.schedule.current        -- idx, which line of the schedule we're on

    if defines.train_state.wait_station == event.train.state
    then
        storage.arrivals[event.train.id] = idx              -- save idx of where this train last arrived
    else

        local nbr = #event.train.schedule.records   -- number of stations in this train's schedule
        local lw = storage.arrivals[event.train.id]         -- idx of where this train last waited at a station

        if nbr > 1 and                                                  -- at least two stops in the entire schedule
           lw and                                                       -- true if we know the station idx where this train last waited
           (defines.train_state.no_path == event.train.state or         -- true if NO_PATH
            defines.train_state.destination_full == event.train.state)  -- true if DESTINATION_FULL
        then -- we might want to skip this stop

            local reachable = false
            local target = idx                                          -- idx of station where this train presently wants to go
            local name = event.train.schedule.records[target].station   -- name of station where this train presently wants to go
            local tmp = event.train.schedule.records[target].temporary  -- true if presently wanted station is a temporary stop
            local at_start = (pat == string.sub(name,1,1))              -- true if start of station name matches our pattern
            local at_end = (pat == string.sub(name,-1,-1))              -- true if end of station name matches our pattern
            local optional = at_start or at_end                         -- true if wanted station is optional
            local wrapped = (target == lw)                              -- true if we wrap around to where we are now

            while optional and not wrapped and not reachable and not tmp do
                if dbg then game.print("skipping optional station " .. name) end
                target = (target+1) > nbr and 1 or (target+1)           -- this is where we try next

                event.train.go_to_station(target)
                reachable = event.train.recalculate_path(true)

                name = event.train.schedule.records[target].station     -- name of station where this train presently wants to go
                tmp = event.train.schedule.records[target].temporary    -- true if presently wanted station is a temporary stop
                at_start = ('?' == string.sub(name,1,1))                -- true if start of station name matches our pattern
                at_end = ('?' == string.sub(name,-1,-1))                -- true if end of station name matches our pattern
                optional = at_start or at_end                           -- true if wanted station is optional
                wrapped = (target == lw)                                -- true if we wrap around to where we are now
            end

            -- SHEMP EDIT: Add a temp stop to alleviate schedule oscillation
            if wrapped then
                local sched = event.train.get_schedule()
                sched.add_record{
                    index = {schedule_index = target + 1},
                    station = name,
                    temporary = true,
                    wait_conditions = {{
                        type = "time",
                        ticks = 300,
                    }},
                }
                event.train.go_to_station(target + 1)
            end
        end
    end
end

----------------------------------------------------------------

local function initStorage()
    if not storage.arrivals then
        storage.arrivals = {}
    end
end

script.on_init(initStorage)
script.on_configuration_changed(initStorage)

script.on_event(defines.events.on_train_changed_state, myTrainChangedStateHandler)

----------------------------------------------------------------
----------------------------------------------------------------

IMPORTANT: You need to increase the mod version before loading this or it will crash.

Events are fine to declare at the top-level of the code, so I got rid of on_load. Any mutable global variables need to go into "storage", and you need to initialise it if it doesn't exist yet.

I've written initStorage() to be able to load an old world, but in the future you could just do:

script.on_init(function()
  storage.arrivals = {}
end)

I've also written a little part (marked with SHEMP EDIT) that makes a temporary stop if the schedule has wrapped around.

You may also want to take a look at the code in this similar mod if you get stuck again. Good luck!

Thank you sir, madam, or other! I have re-released v1.0.1 with the MP addition for persistent storage and updated the changelog and description. One of my iterations was mildly close to this, but I was calling script.on_event(...) after initStorage and that seemed to break things. Thanks for providing the magic sauce.

I appreciate the patch for the schedule oscillation, but an unwritten rule I had given myself when writing the mod is that I didn't want to ever alter the schedule in any way. However, I can see that some folks might prefer that change to avoid such trains being active every other tick. (It should only happen in the degenerate case of a train with optional stops finding all but one stops are closed, so it shouldn't impact any vanilla trains.) Once I resolve the TODO to make certain options user-controlled, I will add your patch in with such a control.

Actually, thinking about it, the train's default behaviour (without your mod installed) is to just wait at the stop, right? So my approach was wrong from the start.

You just need to not send the train anywhere if the schedule loops back on itself. But then how will the train wake up again? Hmm..

EDIT: Ah, but you're using go_to_station() to work out if the train can path there. Seems like you'll want to look into LuaTrainManager::request_train_path.

The oscillation is just a state change, and it does have the {dubious?} advantage that the train will wake up automatically if any conditional stop opens up. I suspect querying request_train_path(...) would be more expensive on average than simply advancing to the next station as implemented, except for the degenerate case of all stops closed when they'd be equal.