Algorithms

Given a specification and a plant we want an algorithm to synthesize a supervisor. This supervisor should be both non-blocking and controllable. Non-blocking guarantees that some desired state can always be reached, so that at least one of the defined (sub-)tasks can always be completed. Controllability guarantees, on the other hand, that the supervisor can always keep the plant within the limits set by the specification. The closed-loop system will never reach a state such that an uncontrollable event can make it exit the specification. These two properties are resolved by two different algorithms, the trimming algorithm managing the non-blocking and the extension algorithm handling the controllability. However, these algorithms interfere with each other, so that trimming may impair controllability, and vice versa. Therefore, the two algorithms must be run iteratively, first one and then the other, until they return the same result. It can be proven, see for instance Fabian (1995), that if the algorithms remove[1] exactly those states that break the respective property, and no more, then their result will be the largest, in number of states, supervisor possible. Since we have knowledge about the structure of the initial candidate, that is, the  specification refines the plant, we know that the supervisor will always be the unique minimal restrictive supervisor.

Non-blocking

For the supervisor to be non-blocking, that is, such that , we know that it has to be trim. This follows from the fact that the specification, and hence any of it sub-automata, always refines the plant, so that . When S is trim, and only then (assuming S accessible), will it hold that . Thus, the supervisor is non-blocking if and only if it is trim. Therefore, what we need is to calculate a trim sub-automaton of a given automaton. To do this, we have to find the set of co-accessible states. Remember that we always assume a given automaton to be accessible, so that trim is equivalent to co-accessibility.

However, we do not just want to find any trim sub-automaton, the null automaton is, for instance, always a valid solution to the problem. We want to find the largest possible such sub-automaton. This we do, of course, if we find the largest co-accessible subset of the state-space.

Two more things deserve to be pointed out before we examine trimming algorithms. First, these algorithms work in isolation from the plant. The task is to trim a given automaton, and trimming is a local property in the sense that it has nothing to do with any other automaton that the trimmed one is to interact with. Generally, the interaction of two trim automata is not trim. However, in our case, we know that the interaction will have the same behavior as the trimmed automata. This is in contrast to the controllability property, which is inherently related to the plant; see below.

The other thing to point out is that we speak of trimming algorithms in plural. This is no accident, we will describe two algorithms. The reason for this is that it is not immediately evident how to treat forbidden states that the given automaton may have. Should we, or should we not , consider those when calculating the trim sub-automaton? In the supervisor calculation, it turns out that we should always do this, but since a specific optimization will be simpler to explain, we will give both an algorithm where we do not consider the forbidden states and one where we do. Of course, the second algorithm will revert to the first in the “degenerate” case that we feed it an automaton with no forbidden states. It will be less efficient, though, due to the check for forbidden states, which in this case would always turn out negative.

Algorithm I

Pre-condition: A given (accessible) automaton  with no forbidden states.

Post-condition: An automaton  the non-forbidden counterpart of which is the largest trim sub-automaton of A.

1.    Let . Let .

2.    For each state ,
if  such that , let .

3.    If , then let  and let . Repeat step 2.

4.    Let .

Note that  has been used above to denote assignment.

The algorithm begins in step 1 by defining two sets,  and Q, that will be used to hold intermediate results.  is set to the set of marked states . Next, the algorithm searches through those states not in  and adds to Q those states from which  can be reached by one transition. If any such state is found, that is, if Q is non-empty in step 3,  is extended by the found states and step 2 is reiterated. Else, if no new states were found, the algorithm terminates and the set of forbidden states are those that are not in .

The algorithm is careful to check only states not already added to . Thus, the number of states checked in each iteration of Step 2 decreases. This, together with there only being a finite number of states, means that the algorithm will always terminate.

This is far from the most efficient algorithm possible. A (recursive) breadth first search would surely be much more efficient[2]. However, the algorithm is intuitive and easily explained and implemented.

Algorithm I does not take a given set of forbidden states into account. This is straightforwardly included, though.

Algorithm II

Pre-condition: A given (accessible) automaton  where .

Post-condition: An automaton  the non-forbidden counterpart of which is the largest trim sub-automaton of A.

1.    Let ,  and .

2.    For each state ,
if  such that , let , else
if  either  or , let .

3.    If , then let  and let . Repeat step 2.

4.    Let .

The main difference between algorithms I and II is that Algorithm II tests a smaller part of the state-space on each iteration. In Step 2, only states that are not already determined to be co-accessible or forbidden are tested. When a state is found that either only reaches already forbidden states, or does not reach any state (that is, is a deadlock) then this state is added to the set of forbidden states. Note that in Step 4 is set to the states not in . This is not necessarily the same as the states in , since the tests in Step 2 are not exhaustive. There may be states for which both tests are false. One example is two non-forbidden states that form a loop but from which no marked state is reachable, this is usually called a livelock. Since these states are neither co-accessible, no marked states are reachable, nor deadlocked, the other state can always be reached, nor only reaching forbidden states, these states are in neither set. However, all states that are not co-accessible must be forbidden, thus .

Trimming

              

The two trimming algorithms are exemplified by trimming the two automata above. The left one has no forbidden states so Algorithm I apply, while the right one requires Algorithm II.

For , Algorithm I sets  and . In Step 2, each of the states in  is tested for reaching into . Both q1 and q2 pass this test and are added to Q and consequently in Step 4 to . Thus, . Next, the states in  are subject to the same test, which only q0 passes so that . Finally, no states pass the test of Step 2, which makes  terminating the algorithm with . As a result, the trim sub-automaton consists of the  and the transitions reaching those states.

Algorithm II begins by setting ,  and . Step 2 tests the states in . Of these, q1 passes the first if clause and thus , while q5 is a deadlock state that can therefore be forbidden, so that . The initial state q0 does not pass either test. In Step 3,  so that . Returning to Step 2, the state q0 is the only one tried for reaching , which it does so that . Thus, . On the next iteration, no states are added to Q, and the algorithm terminates. Finally, . Not surprisingly, q2 belongs to the set of forbidden states, which it did initially, but note that it has not been tested during any iteration. Furthermore,  was only tested once by Algorithm II, while it was tested three times by Algorithm I.

Controllability

To guarantee the controllability property we want a sub-automaton such that all states are controllable. We can assume that a number of uncontrollable states have already been identified and marked as forbidden. Note that this set is in addition to any explicitly forbidden states that may have been specified. There is no reason to treat these two sets of forbidden states differently. Some states are explicitly forbidden by the specification, while others are forbidden because they are uncontrollable. In , both these types of states are designated in the same way; they are forbidden. Thus, this algorithm will be called the extension algorithm, since it extends the set of forbidden states by adding to it those states that reach forbidden states by traces of uncontrollable events.

Again, we do not want to find just any controllable sub-automaton, but the one with the largest number of states. The null automaton is by definition always a controllable sub-automaton of any automaton, but not very useful. Thus, we aim to find the unique largest one, the supremal controllable sub-automaton. We can note that controllability is a localproperty in the same sense as was given for the trim sub-automaton above. Though controllability is always related to the plant, in the way we generate our initial supervisor candidate this relationship is weakened. Once we have identified the uncontrollable states, we can work isolated from the plant, only classifying events of the given automaton. In essence, all algorithms say “here is an automaton with a set of designated states, find all states that reach these by such-and-such traces”. For the trimming algorithms, “such-and-such” means “non-forbidden” while for the controllability algorithm it means “uncontrollable”.

Algorithm III

Pre-condition: A given (accessible) automaton  with a set of uncontrollable events  and where a subset of  are the uncontrollable states.

Post-condition: An automaton  which is the largest controllable sub-automaton of A.

1.    Let  and .

2.    For all states ,
if  such that , then .

3.    If  then let , and

4.    Let .

The algorithm starts with a set of forbidden states  equal to the given forbidden states, and then tests all states not already forbidden. The test is positive if there exists some uncontrollable transition from the tested state to the set of forbidden states. If so, the set of forbidden states is extended with this state. This continues until no more such states can be found, and then the set of forbidden states is exactly those that have been found. In this way, the algorithm extends the set of forbidden states on each iteration with the states that uncontrollably reach the current forbidden state-set.

Extending

The extension algorithm is exemplified by its working on the above automaton. There are only two (types of) events, controllable c, and uncontrollable uc. The q7 state is forbidden, presumable because it is uncontrollable. Thus, . For all states but q7 it is checked in Step 2 whether they reach q7 via uncontrollable events. Only q5 satisfies this, and is added to Q and thus to  in Step 3 so that . Iterating back to Step 2, now the state q3 is found and added to . On the next iteration, no new states are added, thus  and the algorithm terminates with . What remains is the sub-automaton consisting of the states  with their transitions. Since the transitions from q2 and q4 to q3 and q5, respectively, are controllable, the supervisor can prevent the closed-loop system from transiting to those states. Note that, the transition from  to , though controllable, need not be prevented. The  state is controllable and safe for the system to reach with respect to controllability. On the contrary, for the result to be the supremal controllable supervisor, the  to transition must be allowed. That  is a blocking state does not concern the controllability.

Supervisor Synthesis

When synthesizing a supervisor we start with an initial supervisor candidate  and executes the previous algorithms iteratively until their results are equivalent. Each time it is subjected to one of the algorithms, au sub-automaton of it is returned. The sub-automata can be enumerated and we iterate the algorithms until . Here  denotes subjecting sub-automaton  to the trimming algorithm, and likewise  denotes the extension algorithm. The fact that the forbidden states are treated uniformly by Algorithm III whether they are forbidden explicitly or because of uncontrollability makes the algorithm useful in relation to the trimming algorithm. The output from the trimming algorithm is an automaton with the non co-accessible states forbidden. Of course, these states should not be reached uncontrollably in the closed-loop system, otherwise the non-blocking property is violated even though the controllability property is not necessarily. That is, we may still remain within the specification but are no longer able to reach some marked state. Therefore, it is of interest to avoid these states while still not breaking the controllability. This is achieved automatically by Algorithm III, precisely because it does not care about the type of the forbidden states. In some cases, the trimming algorithm is the only way forbidden (and from those, uncontrollable) states enter the specification.

Assume, for instance, that we have a static specification that only specifies a marked state for the plant. In that case, we know that  is an automaton equivalent to the plant except for the marked state. Though the plant is always non-blocking (since it has no marked states) the specification is not necessarily so. However,  is controllable with respect to the plant since they are practically the same automaton. The trimming algorithm may forbid states though, so that the non-blocking automaton is no longer controllable. This is illustrated in the Extending example above, assuming that the specification is simply the marking of the initial state and the plant is the actual automaton.

Supervisor Algorithm

Pre-condition: A total specification  such that it refines .

Post-condition: A non-blocking and controllable, minimally restrictive supervisor , guaranteeing fulfillment of the specification.

1.    Let

2.    Let

3.    Let

4.    If , let  and go to 2

5.    Let

During the above algorithm, smaller and smaller sub-automata of  (more and more forbidden states) are calculated as the index i increases. Eventually, a fix-point will be reached, where  and  will return the same automaton that was given to them. In the worst case, this will happen when the null automaton is attained. When such a fix-point is reached the supervisor algorithm terminates returning the automaton at the fix-point. This is the minimally restrictive non-blocking supervisor that we are looking for.

Note that for the output of Algorithm III to be usable as input to Algorithm II, the extension algorithm must “unmark” any marked states that it forbids. Otherwise, the pre-condition of Algorithm II may be violated, some states may be both marked and forbidden. Of course, a convention saying that “forbiddenness” dominates “markedness” would also solve this.

Supervisor Algorithm

              

The supervisor algorithm will be exemplified by its operation on the S0 specification above. The q5 state is forbidden, presumably because it is uncontrollable, but it could be for any reason. The transitions with small handles one them are controllable, the others are uncontrollable. The encircled states are marked, and q0 is the initial state. Applying the trimming algorithm to S0 results in q4 being forbidden, since this state reaches only already forbidden states.

               

Next, the extension algorithm calculates S2. Since e is an uncontrollable event and q4 is forbidden, q3 must also be forbidden for S2 to be controllable. However, d is controllable so the closed-loop system can be hindered from reaching the forbidden state q3. Since S2 is not equal to S1, the trimming algorithm is applied to S2 again, resulting in S3, which is the same as S2. Note that, since q3 is forbidden, q7 is actually an unreachable state that breaks our assumption about accessible automata. The trimming algorithm as given above does not mark this as forbidden, since it is by definition co-accessible. However, all such states are unreachable and have essentially no impact on the closed-loop system. They may be important, though, in the case where no usable supervisor exist, that is, when the initial state becomes forbidden. Then, the presence of non-reachable states may give hints as to what the problem is and how the specification or plant may be altered for a useful result to exist. This is illustrated by the synthesis of the third supervisor for the two-user problem, below.

Purging the final result will “physically” remove the forbidden and the non-reachable states. Thus, the supervisor is S4 above. Note that its alphabet is the same as before, the same as the plant alphabet. Otherwise, the d event would not be controlled.

Efficiency Considerations

We always want our algorithms to be as efficient as possible[3]. By efficient we usually mean “as fast as possible”. Nowadays, memory consumption is not the important factor it once was, most computers have large amounts of memory. The factor of greatest importance is the speed with which our algorithms work; especially in our case, where the state-space explosion problem is very evident. Storing an automaton with millions of states is no problem but iterating through all those million states is very time-consuming. Though the examples in this course are small, real-life applications are usually vastly more extensive. Thus, any optimization we can make of our algorithms to make them more efficient is valuable.

There are many optimizations available. Most of them avoid using an automata representation at all, but since automata are a very powerful way to describe the algorithms and the theory, we feel that for educational purposes such optimizations should be avoided. Note also that it has been proven (see Kumar (1995)) that, in general, any algorithm must search through the entire state-space in whatever way it is represented to calculate the optimal supervisor.

One obvious optimization is with regard to the way we use the trimming algorithm. We know that the trimming algorithm will be used together with an algorithm managing the controllability property. If we can do, at least some work in that direction when trimming, this may be advantageous. As the trimming algorithms above search through the state-space, they forbid states with certain properties. We can observe that, as the algorithms above do their work they do not take into account the states that they forbid with respect to controllability. However, there is no practical reason for this. It is a straightforward operation to check if the states reaching the newly forbidden state do so via an uncontrollable event. If this is the case, than that state must also be forbidden. This means that we forbid more than what would be necessary to guarantee the non-blocking property, but less than what would guarantee non-blocking and controllability. Thus, some of the controllability work could be done while trimming.

Another problem is whether we should run the extension or the trim algorithm first in the supervisor algorithm. As written, the trimming algorithm is run first but there are no technical reasons for this. The two algorithms could be run in any sequence. The best sequence depends on the specification. If the specification is initially controllable then the extension algorithm will return the exact same automaton, and similarly will the trimming algorithm return the same automaton if it is initially trim. For a specification that contains no forbidden states (and hence is controllable) it is advantageous to trim first. This would (probably) include some forbidden states for the extension algorithm to work on.

The trimming and extension algorithms do not guarantee that their results are accessible. As we saw in the example of the supervisor algorithm, the algorithms may generate a non-accessible automaton. Obviously, this breaks our assumption about all automata being accessible[4]. We allow ourselves to break this assumption for efficiency. Removing forbidden and non-reachable states, that is, purging, is an expensive operation that we want to do that as seldom as possible. Since the algorithms are used mainly to implement the supervisor algorithm, we do not routinely include the purging in them but only at the end of the supervisor algorithm. In this way, we again consider efficiency to some degree. If either of the algorithms is to be used on its own, a purging is necessary to guarantee an accessible supervisor.

There is another reason for not purging the result routinely. This reason would also contradict the purging in the supervisor algorithm, and is illustrated in the synthesis of the third supervisor for the two-user problem below. In some cases when there does not exist a usable supervisor for the given specification and plant, the non-accessible parts of the automata resulting from the supervisor algorithm may hint at the reasons why. From the result it can sometimes be deduced how the specification or the plant should be changed to be controllable. This could result in a rebuild of the physical process, or a weakening[5] (but not strengthening) of the specification.



[1] Actually, in our case states will not be removed, but simply marked as forbidden. This is much more efficient implementation-wise since it avoids any memory management implied by “physically” removing states. When the algorithm has terminated, a single purge will generate the supervisor.

[2] This would be similar to checking for traces reaching from q into Qco.

[3] However, in this case we emphasize ease of understanding. Efficiency is only considered as long as it does not significantly impair intuitiveness.

[4] Actually, it does not. Our claim was that any given automaton is accessible. The results from the algorithm are not given, they are manipulations of given state-machines.

[5] Weakening of the specification can sometimes be done automatically, by calculating the infimal controllable super-language of the specification. See Lafortune (199x).