Knowledge Space Theory


Knowledge space theory is a formal theory started in the eighties by Jean-Claude Falmagne and Jean-Paul Doignon, for the assessment of knowledge in humans beings. The initial objective was to construct "an efficient machine for the assessment of knowledge" (Doignon ,Falmagne 1985). Nowadays, with the diffusion of the computers, this "efficient machine" has the form of a computer program aimed at assessing knowledge, competence and learning of single individuals in a given field.

Psychometric theory (eg.classical test theory, or item response theory) was developed much earlier than KST, with the aim of assessing knowledge, ability, proficiency, etc producing a numerical score in a given area. In KST instead, the main goal of assessment is to provide a precise description of what the student knows and does not at a given moment.

KST has two main components:

The deterministic component draws upon discrete math theories (including set theory, lattice and order theory, combinatorics), to point out theoretically plausible combinatorial structures for the representation of knowledge and learning.

The probabilistic component brings about the probabilistic methods and models for the empirical validation and for the application of the deterministic models in real (noisy) contexts.

Some formal definitions:

- The Domain of knowledge is set \(Q\) of questions;

- The Knowledge state is the subset \(K \subseteq Q\) of all questions that an individual is able to solve;

- The Knowledge structure is a pair \((Q,\mathcal{K})\) , where \(\mathcal{K}\) is a collection of Knowledge states, containing at least the empty set and \(Q\) ;

- The Response pattern is the subset \(R\subseteq Q\) of the items correcly solved by an individual;

The objective of an assessment in KST consists of identify the Knowledge state of an individual in a given domain of knowledge \(Q\) by asking a few questions as possible. The assessment of knowledge can be accomplish in a probabilistc or deterministc way.In the deterministic approch is assume that a student's response to an item is correct if and only if the student masters it. So the response pattern and the knowledge state are the same.

According to the probabilistc approch in real situation for each item \(q \in Q\),two types of errors can occurs.

A Careless error occurs when item \(q\) belongs to \(K\) but the response is incorrect.

A Lucky guess occurs when item \(q\) doesn't belongs to \(K\) but the response is correct.

In the probabilistic approach of the Basic Local Indipendence Model (BLIM,Falmagne & Doignon, 1988a, 1988b) , the response pattern and the knowledge state need not to be equal.

Because this two types of errors there will be a proability \(\beta_{q}\) of a careless errors and a probability \(\eta_{q}\) of a lucky guess for each of the items \(q\in Q\). The careless error and the lucky guess probabilities are define as \(0\leq \beta_{q},\eta_{q}< 1\).

Let \(r_q\) be a random variable with realizations in \({0,1}\), representing the either correct (1) or incorrect (0) response to item \(q\). Then the conditional probability of observing a correct response to item \(q\) is:

$$P(r_{q}=1|K)= \begin{cases} 1-\beta_{q} & if& q\in K \\ \eta_{q} &if & q\notin K \\ \end{cases} $$

The assessment procedure works iterativley by amministring an item at the time. In each step the procedure selects the new item to amminister, obtains a response from the students and updates the likelihood of the knowledge states. See for further details you can read the documentation.


This ShinyApp is a pedagogical demostration of how adaptive knowledge assessment works in KST. This demostration using a set of 23 items in the domain of fractions. For this set of items a knowledge structure contain 260 knowledge states was developed. The 23 items and the knowledge structure are used by the ShinyApp in conjunction with the assessment procedure described above.

The ShinyApp operates in two different modalities: either interactively or by simulating the responses of a student ( Start the simulation ). In both case is shown how the adaptive assessment work step-by-step.

Three main parameters can be manipulated:

- The lucky guess probabilities and the careless error probabilities . You can tune lucky guess and careless error of the items. Things have made simple by assuming tht all the items as same lucky guess and careless error probabilities. If you set this parameters to zero what you get is a deterministc assessment otherwise the assessment will be probabilistic

- The Stopping criterion . This parameter is the minimum level that the likelihood of a knowledge state must have to stop the assessment procedure.


Procede to the other Panel.


A Short Guide

This ShinyApp allows the user to perform his own assessment several times, using the interactive and the simulation modalities, changing the parameters of the procedure and observing step by step they affect the assessment results both in terms of accuracy and efficiency.


Step 1

After click on the assessment tab you can locate a slider for setting your lucky guess, careless error and stopping criterion parameters in the bottom left corner of the screen. (Please notice that once you have set this parameters you should avoid to change them during the assessment session)

Step 2

At this point you can start the assessment and answer to the first question which is locate in the middle of the screen. What you get is something similare this screenshot.

Step 3

When you finish with the interactive modality you can also experiment the simulation modality.

In first place you need to restarting the assessment by clicking Restart Button

After that you can switch to simulation modality by setting as the students true state of knowledge your own or other knowledge state changing the parameters (lucky guess, careless error and stopping criterion). In the simulation modality, the App simulate the response of a student with the given knowledge state. To be observe that a number of question in the simulation could sometimes be quite larger, to make the simulation faster in this case you can choice a larger number of simulation step for each click of the button Procede with the simulation using the Number of simulate answer input.

Step 4

At the end of the simulation you can always go back to the Step 1 and try other scenario. In the accompaing documentation ( Here the documentation ) you can find a number of questions for you. By finding an answer to eache of this question you should improve your understanding of the fondamental carateristic and meccanism of assessment procedure.



Have Fun!



The solution to the equations must be written in the lowest terms and with the oblique fraction bar (es.\(\frac{2}{4}\) is 1/2).
The solution to the question of the filling boxes must be a list of numbers divide by commas and no spaces (es.\(\frac{2+3}{2}=\frac{\Box+\Box}{2}\) is 2,3).


show with app 


Fun<-function(L_N,n,r_q,carless,lucky,K,termine){
if(n!=0){
 Pq_k=rep(0,260)

 if (r_q==1){
   Pq_k[which(K[,n]==1)]=as.numeric(1-carless[n])
   Pq_k[which(K[,n]==0)]=as.numeric(lucky[n])
 }else{
   Pq_k[which(K[,n]==1)]=as.numeric(carless[n])
   Pq_k[which(K[,n]==0)]=as.numeric(1-lucky[n])
 }
 L_memo<-L_N
 for (i in 1:260)
   L_N[i]=L_memo[i]*Pq_k[i]/sum(L_memo*Pq_k) #updating rule
 
}else{}
 if(max(L_N)>=termine) {
    x<-L_N
  }else{
    L_Kg=NULL
    for(i in 1:23){L_Kg[i]<-sum(L_N[which(K[,i]==1)])}
    d<-which(abs(2*L_Kg-1)==min(abs(2*L_Kg-1))) #Questioning rule
    n<-sample(c(d,d),1)
    x<-c(L_N,n)
}
return(x)
}
display_item<-function(nitem){
    if (nitem==1){
      
     "Item n. 1) $$ \\big(\\frac{1}{3}+\\frac{1}{12} \\big) : \\frac{2}{9}=$$"
      
    }else if(nitem==2){
      
     "Item n. 2) $$ \\big(\\frac{3}{2}+\\frac{3}{4} \\big) \\times \\frac{5}{3}-2=$$"
      
    }else if(nitem==3){
      
     "Item n. 3) $$ \\big(\\frac{5}{6}+\\frac{3}{14}\\big)\\times\\big(\\frac{19}{8}-\\frac{3}{2}\\big)=$$"
      
    }else if(nitem==4){
      
     "Item n. 4) $$ \\big(\\frac{1}{6}+\\frac{2}{9}\\big)-\\frac{7}{36}=$$"
      
    }else if(nitem==5){
      
     "Item n. 5) $$ \\frac{7}{10}+\\frac{9}{10}=$$"
      
    }else if(nitem==6){
      
     "Item n. 6) $$ \\frac{8}{13}+\\frac{5}{2}=$$"
      
    }else if(nitem==7){
      
     "Item n. 7) $$ \\frac{8}{12}+\\frac{4}{15}=$$"
      
    }else if(nitem==8){
      
     "Item n. 8) $$ \\frac{2}{9}+\\frac{5}{6}=$$"
      
    }else if(nitem==9){
      
     "Item n. 9) $$ \\frac{7}{5}+\\frac{1}{5}=$$"
      
    }else if(nitem==10){
      
     "Item n. 10) $$ \\frac{2}{7}+\\frac{3}{14}=$$"
      
    }else if(nitem==11){
      
     "Item n. 11) $$ \\frac{5}{9}+\\frac{1}{6}=$$"
      
    }else if(nitem==12){
      
     "Item n. 12) $$ \\big(\\frac{1}{12}+\\frac{1}{3}\\big)\\times\\frac{24}{15}=$$"
      
    }else if(nitem==13){
      
     "Item n. 13) $$ 2-\\frac{3}{4}=$$"
      
    }else if(nitem==14){
      
     "Item n. 14) $$\\big( 4+\\frac{3}{4}-\\frac{1}{2}\\big)\\times\\frac{8}{6}=$$"
      
    }else if(nitem==15){
      
     "Item n. 15) $$ \\frac{4}{7}+\\frac{3}{4}=\\frac{\\Box}{28}$$"
      
    }else if(nitem==16){
      
     "Item n. 16) $$ \\frac{5}{8}-\\frac{3}{16}=\\frac{\\Box-\\Box}{16}$$"
      
    }else if(nitem==17){
      
     "Item n. 17) $$ \\frac{3}{8}+\\frac{5}{12}=\\frac{\\Box \\times 3 + \\Box \\times 5}{24}$$"
      
    }else if(nitem==18){
      
     "Item n. 18) $$ \\frac{2}{7}+\\frac{3}{5}=\\frac{5 \\times \\Box + 7 \\times \\Box}{35}$$" 
      
    }else if(nitem==19){
      
     "Item n. 19) $$ \\frac{2}{3}+\\frac{6}{9}=\\frac{\\Box}{9}=\\frac{\\Box}{\\Box}$$"
      
    }else if(nitem==20){
      
     "Item n. 20) Find the lowest common multiple of 6 and 8"
      
    }else if(nitem==21){
      
     "Item n. 21) $$ \\frac{7}{11} \\times \\frac{2}{3}=$$"
      
    }else if(nitem==22){
      
     "Item n. 22) $$ \\frac{2}{5} \\times \\frac{15}{4}=$$"
    }else if(nitem==23){
      
     "Item n. 23) $$ \\frac{9}{7} : \\frac{2}{3}=$$"
      
    }
}
risp<-function(L_N,n,carless,lucky,Ki){
  
  random<-sample(1:100,1)/100
  if (Ki[n]==1){
    if(random<=carless[n]){r_g=0}else{r_g=1}
  }else{
    if(random<=lucky[n]){r_g=1}else{r_g=0}
  }
return(r_g)}
library(shiny)
library(rPref)
library(igraph)
library(shinyBS)


shinyServer(function(input, output,session) {
  source("Assesment_function.R", local=TRUE)
  source("Display_item_Function.R", local=TRUE)
  source("Response_function.R", local=TRUE)
    
  K=read.table("Matrix.txt")
  
  carless<-reactive(rep(input$eta,length(K[1,]))  )
  
  lucky<-reactive(rep(input$beta,length(K[1,])))
  
  termine<-reactive(input$termine)
  
  #INITIALIZING REACTIVE VALUES
  L_Kg<-reactiveValues()
  L_N<-reactiveValues()
  n<-reactiveValues()
  r_q<-reactiveValues()
  x<-reactiveValues() 
  count<-reactiveValues()
  color<-reactiveValues()
  ver<-reactiveValues()
  Ki<-reactive(K[as.numeric(input$KState),])
  
  pref<-high(V1)*high(V2)*high(V3)*high(V4)*high(V5)*high(V6)*high(V7)*high(V8)*high(V9)*high(V10)*high(V11)*high(V12)*high(V13)*high(V14)*high(V15)*high(V16)*high(V17)*high(V18)*high(V19)*high(V20)*high(V21)*high(V22)*high(V23)
  
  colrs<-c("grey","yellow","orange","red")
  
  btg<-get_btg(K,pref, use_dot = FALSE)
  color$a<-rep(0,length(K[,1]))
  
  #RESTART SETTING VALUE WHEN CHANGE SITUATION
  observeEvent(input$Restart,{
    L_N$a<-rep(1/260,length(K[,1]))
    n$a<-0
    L_Kg$a<-rep(0,length(K[1,]))
    count$a<-0
    x$a<-Fun(L_N$a,n$a,r_q$a,carless(),lucky(),K,termine())
    output$feedback_I<-renderUI("")
    output$feedback_C<-renderUI("")
    },ignoreNULL = FALSE,ignoreInit = FALSE)

  r_Correct=reactive(read.table ("Resp.txt"))
  answ<-eventReactive(input$Go,{r_Correct()[n$a]==input$answer})
  
  #ASSEMENT'S START
  observeEvent(input$Go|input$DoItAgain, {
   
     Condition<-reactive (input$simulation==TRUE)
     if(Condition()){
       ###SIMULATION
       #DEFINITION OF A STATE OF KNOWLEDGE
       
       
         for(i in 1:as.numeric(input$N_cycle)){
           if(length(x$a)!=260){
           L_N$a<-x$a[-261]  
           n$a<-x$a[length(x$a)]
           r_q$a<-risp(L_N$a,n$a,carless(),lucky(),Ki())
           x$a<-Fun(L_N$a,n$a,r_q$a,carless(),lucky(),K,termine())
           count$a<-count$a+1
         }else break
           }
       if(input$N_cycle==1){item_time<-n$a}else{item_time<-n$a}
       
       L_N$a<-x$a[-261] 
       
       for(i in 1:23){L_Kg$a[i]<-sum(L_N$a[which(K[,i]==1)])}
       
       output$hist2<-renderPlot({
        
         barplot(as.numeric(Ki()), col="red",ylim=c(0,1.2),names.arg=c(1:23),xlab = "Items",ylab="Probability", main = "Probability of response correct")
         barplot(L_Kg$a,ylim=c(0,1),add=TRUE, names.arg=c(1:23))
         legend("top",legend=c("Theoretical","Estimate"),fill=c("red","grey"),ncol=2,bty="n", xpd = TRUE)
         args.legend = list()
         })
       
       kstate<-which(K[which.max(L_N$a),]==1)
       
       output$hist<-renderPlot(barplot(L_N$a,ylim=c(0,1),xlab = "Knowledge States",ylab="Likelihood", main = "Likelihood of the KS" ))
       
       if(item_time!=0){
         if(r_q$a==1){
           output$feedback_C<-renderUI(
             print(paste0("The answer at the item n.",item_time," was CORRECT")))
           output$feedback_I<-renderUI("")}
         else{output$feedback_I<-renderUI(
           print(paste0("The answer at the item n.",item_time," was INCORRECT")))
         output$feedback_C<-renderUI("")}
       }else{}####
       
       ##HASSE DIAGRAM  
       
       V(btg$graph)$weight<-L_N$a*70*(1-max(L_N$a)+min(L_N$a))
       color$a[which(L_N$a<0.05)]<-"grey"
       color$a[which(L_N$a>=.05 & L_N$a<.15)]<-"yellow"
       color$a[which(L_N$a>=.15 & L_N$a<.5)]<-"orange"
       color$a[which(L_N$a>=.5)]<-"red"
       V(btg$graph)$color<-color$a
       ver$a<-rep(NA,260)
       ver$a[which.max(L_N$a)]<-which.max(L_N$a)
       
       output$hasse<-renderPlot({
         plot(btg$graph,layout=btg$layout,vertex.label=ver$a,vertex.color=V(btg$graph)$color,vertex.size=V(btg$graph)$weight,edge.arrow.size=0.1, margin=-0.1,edge.lty=3,vertex.label.color="black",vertex.label.dist=1.5  )
         legend(x=-1.5, y=1.2, c("L(K) < 0.05",expression("0.05 " <= " L(K) < 0.15"), expression("0.15 " <= " L(K) < 0.5"),"L(K) > .5"), pch=21,   pt.bg=colrs, pt.cex=2, cex=.8, bty="n", ncol=1)
       })

       
       if(max(L_N$a)>=termine()){
         output$State<-renderUI({y<-paste0("Your Knowledge state is currently the number; ",which.max(L_N$a),
                                           " and your state K={",paste(which(K[which.max(L_N$a),]==1), collapse = ","),"}")
         HTML(y)
         })
         output$question<-renderUI({y<-paste0("<strong>The assessment is ended with <strong>",count$a,"<strong> questions <strong>")
         HTML(y)
         })
       }else{
         output$question<-renderUI(print(paste0(count$a," questions were asked!")))
         currentstate<-as.character(which(L_N$a==max(L_N$a)))
         output$State<-renderUI({
           if(length(currentstate)==260){y<-paste0("All the states have the same probability")} else{y<-paste0("The state(s) of knowledge(s) with higher probability is/are currently the number(s); ",paste(currentstate, collapse = " "))}
           HTML(y)
         })}
       
     }else{
       
       if(n$a!=0){ 
         
         if(answ()==TRUE){r_q$a=1} else{r_q$a=0}
         
       }else{}
       
       item_time<-n$a
       x$a<-Fun(L_N$a,n$a,r_q$a,carless(),lucky(),K,termine())
       
       if (length(x$a)!=260){
         n$a<-x$a[length(x$a)]
         L_N$a<-x$a[-261]
         count$a<-count$a+1
       }else{n$a=-1
       L_N$a<-x$a
       }

       updateTextInput(session, "answer", value = "")
       
       for(i in 1:23){L_Kg$a[i]<-sum(L_N$a[which(K[,i]==1)])}
       
       output$hist2<-renderPlot({
         barplot(L_Kg$a,ylim=c(0,1), names.arg=c(1:23),xlab = "Items",ylab="Probability", main = "Probability of response correct")
       })
       
       item<-reactive(
         display_item(n$a)
       )
       
       output$textitem<-renderUI(withMathJax(helpText(item())))
       
       #OUTPUT CORRECT OR WRONG ANSWER
       if(item_time!=0){
            if(r_q$a==1){
         output$feedback_C<-renderUI(
         print(paste0("The answer at the item n.",item_time," was CORRECT")))
         output$feedback_I<-renderUI("")}
         else{output$feedback_I<-renderUI(
         print(paste0("The answer at the item n.",item_time," was INCORRECT")))
         output$feedback_C<-renderUI("")}
       }else{}####
       
       output$hist<-renderPlot(barplot(L_N$a,ylim=c(0,1),xlab = "Knowledge States",ylab="Likelihood", main = "Likelihood of the KS"))

       ##HASSE DIAGRAM
       V(btg$graph)$weight<-L_N$a*70*(1-max(L_N$a)+min(L_N$a))
       color$a[which(L_N$a<0.05)]<-"grey"
       color$a[which(L_N$a>=.05 & L_N$a<.15)]<-"yellow"
       color$a[which(L_N$a>=.15 & L_N$a<.5)]<-"orange"
       color$a[which(L_N$a>=.5)]<-"red"
       V(btg$graph)$color<-color$a
       ver$a<-rep(NA,260)
       ver$a[which.max(L_N$a)]<-which.max(L_N$a)
      
       output$hasse<-renderPlot({
         plot(btg$graph,layout=btg$layout,vertex.label=ver$a,vertex.color=V(btg$graph)$color,vertex.size=V(btg$graph)$weight,edge.arrow.size=0.1, margin=-0.1,edge.lty=3,vertex.label.color="black",vertex.label.dist=1.5  )
         legend(x=-1.5, y=1.2, c("L(K) < 0.05",expression("0.05 " <= " L(K) < 0.15"), expression("0.15 " <= " L(K) < 0.5"),"L(K) > .5"), pch=21,   pt.bg=colrs, pt.cex=2, cex=.8, bty="n", ncol=1)
       })
       
       kstate<-list(as.character(which(K[which.max(L_N$a),]==1)))
       
       if(max(L_N$a)>=termine()){
         output$question<-renderUI({y<-paste0("<strong>The assessment is ended with <strong>",count$a,"<strong> questions <strong>")
         HTML(y)
         })
         output$State<-renderUI({y<-paste0("Your Knowledge state is currently the number; ",which.max(L_N$a),
                                           " and your state K={",paste(which(K[which.max(L_N$a),]==1), collapse = ","),"}")
         HTML(y)
         })
       }else{
         output$question<-renderUI(print(paste0(count$a," questions were asked!")))
         currentstate<-as.character(which(L_N$a==max(L_N$a)))
         output$State<-renderUI({
           if(length(currentstate)==260){y<-paste0("All the states have the same probability")} else{y<-paste0("The state(s) of knowledge(s) with higher probability is/are currently the number(s); ",paste(currentstate, collapse = " "))}
         HTML(y)
         })}
     }
 
    
},ignoreNULL = TRUE)
  
  
})
library(shiny)
library(rPref)
library(igraph)
library(shinyBS)

shinyUI(fluidPage(
  includeScript("../../../Matomo-tquant.js"),
  withMathJax(),
  navbarPage("KAA",
             tabPanel("Introduction",
                      fluidRow(
                        column(6,h2("Knowledge Space Theory")),
                        column(6,a(img(src="tquant100.png",align="left" ,width="60%"),href="https://tquant.eu/"))),
                        fluidRow(
                          column(12,
                                 br(),
                        p('Knowledge space theory is a formal theory started in the eighties by Jean-Claude Falmagne and Jean-Paul Doignon, 
                        for the assessment of knowledge in humans beings. 
                          The initial objective was to construct "an efficient machine for the assessment of knowledge" (Doignon ,Falmagne 1985). 
                          Nowadays,  with the diffusion of the computers, this "efficient machine" has the form of a computer program aimed at assessing knowledge, 
                          competence and learning of single individuals in a given field.'),
                        p('Psychometric theory (eg.classical test theory, or item response theory) was developed much earlier than KST,
                          with the aim of assessing knowledge, ability, proficiency, etc producing a ', strong('numerical score'),
                          'in a given area. In KST instead, the main goal of assessment is to provide a precise description of '
                          ,strong( 'what the student knows'),' and does not at a given moment.'),
                        p('KST has two main components:'),
                        p(strong('The deterministic component'),' draws upon discrete math theories (including set theory, lattice and order theory, combinatorics),
                          to point out theoretically plausible combinatorial structures for the representation of knowledge and learning.'),
                        p(strong('The probabilistic component'),' brings about the probabilistic methods and models for the empirical validation and
                          for the application of the deterministic models in real (noisy) contexts.'),
                        p('Some formal definitions:'),
                        div(p("- The ",strong('Domain of knowledge'), 'is set ', '\\(Q\\)',' of questions;')),
                        div(p("- The ", strong('Knowledge state'), 'is the subset ', '\\(K \\subseteq Q\\)',' of all questions that an individual is able to solve;')),
                        div(p("- The",strong('Knowledge structure'), 'is a pair ', '\\((Q,\\mathcal{K})\\)',', where ', '\\(\\mathcal{K}\\)',' is a collection of Knowledge states, containing at least the empty set and ', '\\(Q\\)',';')),
                        div(p("- The",strong('Response pattern'),'is the subset \\(R\\subseteq Q\\) of the items correcly solved by an individual;')),
                        p("The objective of an assessment in KST consists of identify the Knowledge state of an individual in a given domain of knowledge \\(Q\\) by asking a few questions as possible.
                          The assessment of knowledge can be accomplish in a probabilistc or deterministc way.In the deterministic approch is assume that a student's response to an item is correct if and only if the student masters it. 
                          So the response pattern and the knowledge state are the same."),
                        p("According to the probabilistc approch in real situation for each item \\(q \\in Q\\),two types of errors can occurs. "),
                        div(p("A Careless error occurs when item \\(q\\) belongs to \\(K\\) but the response is ", strong( "incorrect."))),
                        div(p("A Lucky guess occurs when item \\(q\\) doesn't belongs to \\(K\\) but the response is ",strong( "correct."))),
                        div(p("In the probabilistic approach of the Basic Local Indipendence Model (BLIM,Falmagne & Doignon, 1988a, 1988b) , the response pattern and the knowledge state need not to be equal.")),
                        div(p("Because this two types of errors there will be a proability \\(\\beta_{q}\\) of a careless errors and a probability \\(\\eta_{q}\\) of a lucky guess for each of the items \\(q\\in Q\\). 
                              The careless error and the lucky guess probabilities are define as \\(0\\leq \\beta_{q},\\eta_{q}< 1\\).")),
                        div(p("Let \\(r_q\\) be a random variable with realizations in \\({0,1}\\), representing the either correct (1) or incorrect (0) response to item \\(q\\). Then the conditional probability of observing a correct response to item \\(q\\) is:")),
                      
                        div("$$P(r_{q}=1|K)=
                              \\begin{cases}
                              1-\\beta_{q} & if& q\\in K \\\\
                              \\eta_{q} &if & q\\notin K \\\\
                              \\end{cases}
                              $$"),
                        p("The assessment procedure works iterativley by amministring an item at the time. In each step the procedure selects the new item to amminister,
                          obtains a response from the students and updates the likelihood of the knowledge states. See for further details you can read the documentation."),
                        br(),
                        
                        p("This ShinyApp is a pedagogical demostration of how adaptive knowledge assessment works in KST. This demostration using a set of 23 items in the domain of fractions.
                          For this set of items a knowledge structure contain 260 knowledge states was developed.
                          The 23 items and the knowledge structure are used by the ShinyApp in conjunction with the assessment procedure described above."),
                        p("The ShinyApp operates in two different modalities: 
                          either interactively or by simulating the responses of a student (",span("Start the simulation", style = "color:blue"),"). In both case is shown how the adaptive assessment work step-by-step."),
                        

                        
                        p("Three main parameters can be manipulated:"),
                        
                        div(p("- The ",strong("lucky guess probabilities")," and the",strong("careless error probabilities"), ". You can tune lucky guess and careless error of the items. 
                               Things have made simple by assuming tht all the items as same lucky guess and careless error probabilities.
                              If you set this parameters to zero what you get is a deterministc assessment otherwise the assessment will be probabilistic")),
                        div(p("- The ",strong("Stopping criterion"),". This parameter is the minimum level that the likelihood of a knowledge state must have to stop the assessment procedure. ")),
                        br(),
                        p("Procede to the other Panel."),
                        str5<-HTML('<center><img src="fig1.jpg"></center>'),
                        br()
                          
                      
                      ))),
             tabPanel ("A Short Guide",
                       fluidRow(column(2),column(8, h2("A Short Guide"),
                                                 p('This ShinyApp allows the user to perform his own assessment several times, 
                                                   using the interactive and the simulation modalities, changing the parameters of the procedure
                                                   and observing step by step they affect the assessment results both in terms of accuracy and efficiency.'), 
                                                 br(),
                                                 h3("Step 1"),
                                                 p('After click on the assessment tab you can locate a slider for setting  your lucky guess,
                                                   careless error and stopping criterion parameters in the bottom left corner of the screen.
                                                   (Please notice that once you have set this parameters you should avoid to change them during the assessment session)'),
                                                 str5<-HTML('<center><img src="fig3.jpg"></center>'),
                                                 h3("Step 2"),
                                                 p("At this point you can start the assessment and answer to the first question which is locate in the middle of the screen.
                                                   What you get is something similare this screenshot."),
                                                 str5<-HTML('<center><img src="fig4.jpg"></center>'),
                                                 h3("Step 3"),
                                                 p("When you finish with the interactive modality you can also experiment the simulation modality."),
                                                 str5<-HTML('<center><img src="fig5.jpg"></center>'),
                                                 p("In first place you need to restarting the assessment by clicking ",span("Restart Button",style="color:blue")),
                                                 str5<-HTML('<center><img src="fig6.jpg"></center>'),
                                                 p("After that you can switch to simulation modality by setting as the "
                                                   , em("students true state of knowledge")," your own or other knowledge state changing the parameters (lucky guess, careless error and stopping criterion).
                                                   In the simulation modality, the App simulate the response of a student with the given knowledge state. To be observe that a number of question in the simulation could sometimes be quite larger, 
                                                   to make the simulation faster in this case you can choice a larger number of simulation step for each click of the button ",span("Procede with the simulation",style="color:blue")," using the ",span("Number of simulate answer",style="color:blue")," input."),
                                                 str5<-HTML('<center><img src="fig7.jpg"></center>'),
                                                 h3("Step 4"),
                                                 p("At the end of the simulation you can always go back to the ",strong("Step 1"),
                                                   " and try other scenario. In the accompaing documentation 
                                                    (",a("Here the documentation",href="Report.pdf"),") you can find a number of questions for you. 
                                                   By finding an answer to eache of this question you should improve your understanding of the fondamental carateristic and meccanism of assessment procedure."),
                                                 br(),
                                                 br(),
                                                 HTML('<center><h3>Have Fun!</h3></center>'),
                                                 br()
                       ))
                       ),
             tabPanel ("Assessment",
                       fluidRow(
                         column(4,
                                plotOutput("hist2")
                                ),
                         column(4,
                                plotOutput("hasse")
                                ),
                         column(4,
                                plotOutput("hist")
                         )
                       ),
                       fluidRow(
                         column(4,
                                checkboxInput("simulation",span("Start the simulation", style = "color:blue")),
                                sliderInput(inputId="eta", label = "Careless error",
                                            min=0, max=0.5, step=0.01, value = 0),
                                sliderInput(inputId="beta", label = "Lucky guess",
                                            min=0, max=0.5, step=0.01, value = 0),
                                sliderInput(inputId="termine", label = "Stopping criterion",
                                            min=0, max=1.0, step=0.05, value = 0.5)
                         ),
                         column(4, 
                                br(),
                                conditionalPanel("!input.simulation",
                                div("The solution to the equations must be written in the lowest terms and with the oblique fraction bar (es.\\(\\frac{2}{4}\\) is 1/2).", align = "left")),
                                conditionalPanel("!input.simulation",
                                div("The solution to the question of the filling boxes must be a list of numbers divide by commas and no spaces (es.\\(\\frac{2+3}{2}=\\frac{\\Box+\\Box}{2}\\) is 2,3).", align = "left")),
                                br(),
                                conditionalPanel("!input.simulation",
                                                 uiOutput("textitem")),
                                conditionalPanel("!input.simulation",
                                                 textInput("answer",NULL,"")),
                                conditionalPanel("!input.simulation",
                                                 actionButton("Go", "Confirm")),
        
                                br(),
                                
                                conditionalPanel("input.simulation",
                                                 actionButton("DoItAgain", "Procede with the simulation")),
                                div(strong(uiOutput("feedback_C"),style="color:green")), 
                                div(strong(uiOutput("feedback_I"),style="color:red"))
                         ),
                         column(4,
                                conditionalPanel("input.simulation",
                                                 numericInput("N_cycle", "Number of simulated answer", 1, min = 1, max = 1000)),
                                conditionalPanel("input.simulation",
                                                 sliderInput(inputId="KState", label = "Knowledge  state simulated",
                                                             min=1, max=260, step=1, value = 10)),
                                htmlOutput("State"),
                                uiOutput("question"),
                                br(),
                                br(),
                                actionButton("Restart", "Restart assessment")
                                )
                         )
                       )
  )))