Constraint Satisfaction Revisited

Definitions

Recall from last time that a Constraint Satisfaction (CS) Problem is one in which the solution path does not matter and the goal state is required to satisfy a set of consistency conditions.
More specifically, a Constraint Satisfaction Problem is characterized by:

a set of variables {x1, x2, .., xn},
for each variable xi a domain Di with the possible values for that variable, and
a set of constraints, i.e. relations, that are assumed to hold between the values of the variables.

The constraint satisfaction problem is to find, for each i from 1 to n, a value in Di for xi so that all constraints are satisfied.

Example

Search can be made easier in cases where the solution instead of corresponding to an optimal path, is only required to satisfy local consistency conditions.
For example, in a crossword puzzle it is only required that words that cross each other have the same letter in the location where they cross.
Many problems can be stated as constraints satisfaction problems.
A crossword puzzle is a simple example of constraint satisfaction problem:


      1   2   3   4   5
    +---+---+---+---+---+	Given the list of words:
  1 | 1 |   | 2 |   | 3 |		AFT	LASER
    +---+---+---+---+---+		ALE	LEE
  2 | # | # |   | # |   |		EEL	LINE
    +---+---+---+---+---+		HEEL	SAILS
  3 | # | 4 |   | 5 |   |		HIKE	SHEET
    +---+---+---+---+---+		HOSES	STEER
  4 | 6 | # | 7 |   |   |		KEEL	TIE
    +---+---+---+---+---+		KNOT
  5 | 8 |   |   |   |   |
    +---+---+---+---+---+	
  6 |   | # | # |   | # |       The numbers 1,2,3,4,5,6,7,8 in the crossword
    +---+---+---+---+---+       puzzle correspond to the words 
				that will start at those locations.

Representation

A CS problem is usually represented as an undirected graph, called Constraint Graph
The nodes are the variables and the edges are the binary constraints.
Unary constraints can be disposed of by just redefining the domains to contain only the values that satisfy all the unary constraints.
Higher order constraints are represented by hyperarcs.
In this example we restrict our attention to the case of unary and binary constraints.

Example revisited:

We introduce a variable to represent each word in the puzzle. So we have the variables:


	VARIABLE | STARTING CELL | DOMAIN
	================================================
	1ACROSS	 | 1		 | {HOSES, LASER, SAILS, SHEET, STEER}
	4ACROSS	 | 4		 | {HEEL, HIKE, KEEL, KNOT, LINE}
	7ACROSS	 | 7		 | {AFT, ALE, EEL, LEE, TIE}
	8ACROSS	 | 8		 | {HOSES, LASER, SAILS, SHEET, STEER}
	2DOWN	 | 2		 | {HOSES, LASER, SAILS, SHEET, STEER}
	3DOWN	 | 3		 | {HOSES, LASER, SAILS, SHEET, STEER}
	5DOWN	 | 5		 | {HEEL, HIKE, KEEL, KNOT, LINE}
	6DOWN	 | 6		 | {AFT, ALE, EEL, LEE, TIE}

The domain of each variable is the list of words that may be the value of that variable.
For example, variable 1ACROSS requires words with five letters, 2DOWN requires words with five letters, 3DOWN requires words with four letters, etc.
Note that since each domain has 5 elements and there are 8 variables, the total number of states to consider in a naive approach is 5**8 = 390,625.

Binary Constraints

The constraints are all binary constraints:

	1ACROSS[3] = 2DOWN[1]	i.e. the third letter of 1ACROSS must be equal
				to the first letter of 2DOWN
	1ACROSS[5] = 3DOWN[1]
	4ACROSS[2] = 2DOWN[3]
	4ACROSS[3] = 5DOWN[1]
	4ACROSS[4] = 3DOWN[3]
	7ACROSS[1] = 2DOWN[4]
	7ACROSS[2] = 5DOWN[2]
	7ACROSS[3] = 3DOWN[4]
	8ACROSS[1] = 6DOWN[2]
	8ACROSS[3] = 2DOWN[5]
	8ACROSS[4] = 5DOWN[3]
	8ACROSS[5] = 3DOWN[5]

Graphical Representation

Solution Methods

There are three standard solution methods for constraint satisfaction problems:
- Generate-and-Test,
- Backtracking [possibly Dependency Directed]
- Constraint Propagation

Generate and Test

Generate one by one all possible complete variable assignments
For each complete variable assignment, test if it satisfies all constraints.
The program structure is very simple, just nested loops, one per variable.
In the innermost loop, test each constraint.
In most situation this method is intolerably slow.

Backtracking

Order the variables trying to place the variables that are more highly constrained or with smaller ranges first.
This order has a great impact on the efficiency of solution algorithms.
Check constraint satisfaction at the earliest possible time and extend an assignment if the constraints involving the currently bound variables are satisfied.

Example Revisited:

In the crossword puzzle we may order the variables as follows:
- 1ACROSS
- 2DOWN
- 3DOWN
- 4ACROSS
- 7ACROSS
- 5DOWN
- 8ACROSS
- 6DOWN

Then start the assignments:

   1ACROSS 	<- HOSES
    2DOWN 	<- HOSES     => failure, 1ACROSS[3] not equal to 2DOWN[1]
		<- LASER     => failure
		<- SAILS
     3DOWN	<- HOSES     => failure
		<- LASER     => failure
		<- SAILS
      4ACROSS	<- HEEL	     => failure
		<- HIKE	     => failure
		<- KEEL	     => failure
		<- KNOT	     => failure
		<- LINE	     => failure, backtrack
     3DOWN	<- SHEET
      4ACROSS	<- HEEL
       7ACROSS  <- AFT	     => failure
	................................

Chronological Backtracking

As shown above, in Chronological Backtracking, variables are unbound in the inverse order to the the order used when they were bound.
Dependency Directed Backtracking instead recognizes the cause of failure and backtracks to one of the causes of failure and skips over the intermediate variables that did not cause the failure.

Constraint Propagation

We can use information from the constraints to reduce the search space as early in the search as it is possible.
In the crossword example, variables (i.e., node in the constraint graph) are consistent if they satisfy all unary constraints as well as the constraints on the arcs.

Constraint Propagation Algorithm

A solution to a CS problem gives a value ui to each variable xi of the problem. These values satisfy all the constraints of the problem.
In a CS problem with only unary and binary constraints, if we reduce the domains of the variables so that the resulting CS problem satisfies all constraints expressed in the graph, the resulting domains contain all the solutions.
This is the foundation for an efficient and simple Consistency Based Algorithm, AC-3, due to Mackworth, that solves CS problems with only unary and binary constraints.

The AC-3 Algorithm

   Function AC-3 (CS-Problem) is
      Let Q be a set initialized to contain all the arcs of CS-Problem;
	NOTE: since the constraint graph is undirected, each of its
	      edges is represented by two directed arcs.
   begin
      Reduce all the domains of CS-Problem so that they satisfy the 
	 unary constraints;
      While Q is not empty
	 Remove an arc (xi xj) from Q;
	 If constraint-propagation(xi, xj) then
	    If the domain of xi is empty 
	        Return with failure;
	    else
	        Add to Q all arcs (xk xi), with k different than 
		    i and j, that are in the CS-Problem;
      Return with success;
   end AC-3;

The Constraint Propagation Check

   Function constraint-propagation (xi, xj: problem variables) is
      Let Change be a boolean variable initialized to false;
   begin
      For each value u in the domain of xi 
	 find a value v in the domain of variable xj such that 
	     u and v satisfy all the binary constraints of the problem. 
	 If there is no such v, 
	    remove u from the domain of xi and set Change to true;
      Return the value of Change;
   end constraint-propagation;

Upon completion of the AC-3 algorithm we usually have to run some form of the backtracking algorithm to find the solutions, if any.

AC-3 Applied to the Crossword Puzzle

We will write thevariable 1ACROSS as just 1A, 2DOWN as 2D, etc.
The AC-3 algorithm proceeds as follows:

	(xi,xj)	| NEW Di, if changed	| ADDED {xk,xi}
	===================================================================
	(1A 1D)	| 			| 
	(1D,1A) | 			| 
	(1A,2D)	|			|
	(2D,1A) |			|
	(1D,3A) |			|
	(3A,1D)	|			|
	(3A,2D)	|			|
	(2D,3A)	|			|
	===================================================================

AC-3 Example Continued

Here are the steps of the AC-3 algorithm in the crossword example.

	(xi,xj)	| NEW Di, if changed	| ADDED {xk,xi}
	===================================================================
	(1A 2D)	| HOSES,LASER		| (3D 1A)
	(3D,1A) | SAILS,SHEET,STEER	| (4A,3D) (7A,3D) (8A,3D)
	(4A,3D) | HEEL,HIKE,KEEL	| (2D,4A) (5D,4A)
	(7A,3D) | ALE,EEL,LEE,TIE	| (2D,7A) (5D,7A)
	(8A,3D)	| 			|
	(2D,4A)	| SAILS,SHEET,STEER	| (1A,2D) (7A,2D) (8A,2D)
	(5D,4A)	| KEEL,KNOT		| (7A,5D) (8A,5D)
	(2D,7A)	|			|
	(5D,7A)	| KEEL			| (4A,5D)
	(1A,2D)	| 			|
	(7A,2D) | EEL,LEE		| (3D,7A) (5D,7A) 
	(8A,2D)	| HOSES,LASER		| (3D,8A) (5D,8A) (6D,8A)
	(7A,5D)	| 			|
	(8A,5D)	|			|
	(4A,5D)	| HIKE			| (2D,4A) (3D,4A) 
	(3D,7A) |			|
	(5D,7A) |			|
	(3D,8A) | SAILS,STEER		| (1A,3D) (4A,3D) (7A,3D)
	(5D,8A) | 			|
	(6D,8A) | ALE			| 
	(2D,4A) | SAILS			| (1A,2D) (7A,2D) (8A,2D)
	(3D,4A) | STEER			| (8A,3D)
	(1A,3D) | HOSES			| (2D,1A) 
	(4A,3D) |			|
	(7A,3D) | LEE			| (2D,7A) (5D,7A)
	(1A,2D) |			|
	(7A,2D) |			|
	(8A,2D) |			|
	(8A,3D) | LASER			| (2D,8A) (5D,8A) (6D,8A) 
	(2D,1A) |			|
	(2D,8A) |			|
	(5D,8A) |			|
	(6D,8A) |			|
	===============================================================

Solution

Thus the solution is:

	      1   2   3   4   5
	    +---+---+---+---+---+	
	  1 | H | O | S | E | S |	
	    +---+---+---+---+---+	
	  2 | # | # | A | # | T |	
	    +---+---+---+---+---+	
	  3 | # | H | I | K | E |	
	    +---+---+---+---+---+	
	  4 | A | # | L | E | E |
	    +---+---+---+---+---+
	  5 | L | A | S | E | R | 
	    +---+---+---+---+---+
	  6 | E | # | # | L | # |
	    +---+---+---+---+---+