Redirecting Viewlet Stats to your own Server (VB6)

10/30/2013 7:36 AM
You can subscribe to this wiki article using an RSS feed reader.

Overview:

This post describes how to capture the reporting results from a Viewlet and redirect it to your server. This allows you to have full control over the collection and analysis of your Viewlets. This article covers the following versions of ViewletBuilder and ViewletQuiz:

  • ViewletBuilder6 (versions 6.2.8 and above)
  • ViewletBuilder7 (all versions)
  • ViewletQuiz3 (versions 3.2.8 and above)
  • ViewletQuiz4 (all versions)
  • for newer versions, please read this article instead.

If a Viewlet was published with reporting set to “Qarbon Email Reporting System”, it will send XML data to our server that describes the user’s interactions. You can change this behavior to have the stats sent you your own server.

The difficult part is writing the backend program to process the XML and write it to your database. This program could be written in Java, PHP, Python, Perl or any other language used for processing web pages.

Redirect Results to your Server:

ViewletBuilder7 / ViewletQuiz4 Instructions:

The process has been simplified in ViewletBuilder7 and ViewletQuiz4. Here are the steps:

  1. Publish your Viewlet to a folder target. For example "Current Project" or "My Viewlets"
  2. Check the "Email Results To" check box.
  3. Click on the "Advanced" button
  4. Select the "Custom Reporting Server" checkbox
  5. Click OK

Now every time you publish to this target, it will send the XML stats to your custom sever.

ViewletBuilder6 / ViewletQuiz3 Instructions:

Publish your Viewlet to a local folder. My Viewlets for example.

On the reporting options page of the publishing wizard, select Qarbon Email Reporting System.

When the publish wizard completes, click the Browse button to open up the folder where your Viewlet was published.

Open the xxx_viewlet_swf.html file with notepad (Where xxx is the name of your Viewlet)

You will see the following name=value pair in a block of javascript that loads the flash file.

rp=http://services.qarbon.com/client/statsmailer

Replace the url to the Qarbon statsmailer program to a URL to a program on your server that can process the XML results.

For example it will look something like this:

<script language="JavaScript">
<!--
var flashTag = new FlashTag("emalreporttest.swf", "644", "542");
flashTag.setId("emalreporttest"); flashTag.setFlashvars("jsfgid="+jsfgid+"&__variables__="+getParameters()+"&rp=http://www.mycompany.com/servlet/processviewlet");
flashTag.write(document);
//-->
</script>

You may want to also pass a student id along with this Viewlet. For example suppose you have an authenticated website from which the Viewlets are served. You may want to pass the authenticated user’s id along with their results. Simply add the argument student_id= with the student id to your URL. It will then be passed in the XML data.

For example:
rp=http://www.mycompany.com/servlet/processviewlet&student_id=1234

Alternatively, if you compiled the Viewlet with the "Require Viewers to Login" checkbox selected in the publishing wizard, then the username entered here will be passed in the XML data.

Customizing an HTML Frame

If you want to have these changes in multiple projects, and don't want to manually edit the HTML each time, you can create a new HTML Frame within ViewletBuilder and add the "rp" parameter there.

Go to "Project Properties" in ViewletBuilder or ViewletQuiz

Select "HTML Frame".

Click the "+" button to add a new HTML Frame.

Name the frame something like "Custom Reporting Frame"

Double click on the entry named "$VIEWLET_NAME$_viewlet_swf.html"

Look for the line that looks like this:

flashTag.setFlashvars("__variables__="+getParameters()+"$VIEWLET_PARAMETERS$");

Replace "$VIEWLET_PARAMETERS$" with the URL to your server as follows:

flashTag.setFlashvars("__variables__="+getParameters()+"&rp=http://www.mycompany.com/servlet/processviewlet");

Now just select this HTML Frame and select Email Reporting when you want to send the results to your own server.


Process the Results on your Server:
The Viewlet will send the results, for each user, as a series of XML HTTP posts to the specified server. It sends a summary XML report at the beginning and one after every slide in the Viewlet. Breaking the results into a series of reports rather than a single report allows us to capture partial results from a user. With surveys it is not uncommon for a user to answer a few questions and then give up and close the browser. This information can be very useful to capture.

The series of XML reports makes the processing the results a little more challenging. We recommend using a cookie to group all the reports and then process it when the last report is received.

Each XML report has root tag of <report>

Here is a sample of what the first report looks like. It is identified by having a <begin> tag. Comments added in BLUE.

<?xml version="1.0" encoding="UTF-8"?>
<!-- Standard xml header, added in VB / Quiz.
Since: VB 6.2.9 -->

<!-- Each report starts with a report tag. -->
<report reportVersion="2" timestamp="123" request="1093563296">
<!--
reportVersion: Set to 2 for VB5 and ViewletQuiz2 and above.

timestamp:     Client time in milliseconds. Represents the time in milliseconds
               since the start of the Viewlet. Since reports could come in out
               of order due to asynchronous nature of the internet, these
               timestamps can be used to keep the reports in chronological order.
               Since:  VB 6.2.8

request:       A unique number generated by the Viewlet when it is first 
               displayed to the user. It remains the same until the Viewlet is
               restarted.

               The purpose of this is to deal with the case of a user
               having 2 Viewlets open in 2 separate browser tabs. In this case,
               they could both share the same cookie id, and we want to make sure
               we segregate the reporting results.
-->

 
  <!-- The begin tag is added for the first report sent from the Viewlet. Its purpose is
  to provide the user and contain the <content> tag below. -->
  <begin user="andrew"> 
  <!-- user: The user entered from the student_id param or if a login panel -->

      <content
        manifestVersion="3"
        id="-1"
        language="en"
        name="MyViewlet.qvp"
        author="John Doe"
        description="This is a sample viewlet."
        title="MyViewlet"
        application="ViewletBuilder6"
        applicationVersion="6.2.10"
        width="852"
        height="542"
        slideCount="10"
        flash="8"
        passingGrade="65"
        timeLimit="0"
        htmlFile="myviewlet_viewlet_swf.html"
        swfFile="myviewlet.swf"
        thumbnailFile=""
        slidesIncluded="true">
        <!--
        The content section describes meta information about the Viewlet
        manifestVersion: Set to 3 for ViewletBuilder 6 and higher

        id: ViewletCentral specific

        language: The 2 digit ISO language code of the Viewlet

        name: The name of the project file this was generated from

        author: The author name set in the project, via project properties.

        description: Viewlet description set in project properties.

        title: Viewlet title, set in project properties.

        application: The application used to generate the Viewlet.

        width: The width of the Viewlet including ViewletSkin

        height: The height of the Viewlet including ViewletSkin

        slideCount: The number of slides in the Viewlet

        flash: The minimum version of flash this Viewlet requires.

        passingGrade: The % correct required to pass this Viewlet.
        Set via Project Properties > Scoring Options

        timeLimt: Time limit to complete the Viewlet in milliseconds.

        htmlFile: The name of the html file that contains this Viewlet

        swfFile: The flash file containing the Viewlet

        thumbnailFile: The thumbnail image generated for the Viewlet

        slidesIncluded: Set to true for VB 6.2.8 and higher.
        -->

         <!-- The email address to send the response back to -->
         <emailTo value="qarbie@qarbon.com" />

         <!--
         Following there is a tag for each slide in the Viewlet, in the order
         they appear in the Viewlet. Each slide tag has an id attribute which
         will be referenced when reports come in for this slide. Slides can at
         most contain one interactive zone or question. Therefore slides that
         contain no question or interactive zone are represented with a <slide>
         tag. Those with questions are zones are identified by the name of that
         zone or question type, for example <clickZone> or <shortAnswerQuestion>.
         -->

         <!-- Here is an example of the first slide: -->
         <slide id="146" name="Slide 1" duration="1000" />
         <!--
         id: When reports come in for this slide it will be referenced by
         this id.
         name: The name of the slide.
         duration: The time the slide is to be displayed in milliseconds.
         -->

         <!-- The 2nd slide contains a click zone. -->
         <clickZone id="367" name="Slide 2" points="1" timeLimit="0"
            attempts="3" survey="false">
            <clickArea id="368" correct="false" />
         </clickZone>
         <!--
         name: The name of the slide containing this click zone.

         points: The number of points for getting responding to the click zone

         correctly. Set in the Score dialog for the Click Zone.

         timeLimit: The time in milliseconds to answer this zone correctly. Set
         in the Score dialog for the Click Zone. If 0 then there is no timelimit.

         attempts: The number of attempts the user has to respond to this click
         zone correctly.

         survey: True if this is a survey click zone, and therefore doesn't have
         a correct answer.
        
         clickArea id: the id of the click area in the click zone.
         ClickZones only have one area. Click Map's however could have multiple
         areas.

         correct: Not really relevant for Click Zones, since the can only
         have one clickArea its always the correct one.
         -->

         <!-- The 3rd slide contains a key stroke zone. -->
         <keyStrokeZone id="379" name="Slide 4" points="1"
             timeLimit="0" attempts="3" survey="false">
             <keyStroke id="A" />
         </keyStrokeZone>
         <!--
         keyStroke id: The character the user needs to hit to respond correctly.
         -->

         <!-- The 4th slide contains a short answer question fill in the blank -->
         <shortAnswerQuestion id="151" name="What is 1 2" points="1"
            timeLimit="0" attempts="3" survey="false" question="What is 1 + 2">
            <textField id="153" caseSensitive="false">
               <answer value="3" />
            </textField>
         </shortAnswerQuestion>
         <!--
         question: The text of the question being asked.
         caseSensitive: true if the user must enter the text in the proper case.
         answer: The text of the correct answer.
         -->

         <!-- This slide contains a short answer question select from list -->
         <shortAnswerQuestion id="158" name="What is 2 3" points="1"
            timeLimit="0" attempts="3" survey="false" question="What is 2 + 3">
            <comboBox id="160">
               <answer id="1" value="3" correct="false" />
               <answer id="2" value="4" correct="false" />
               <answer id="3" value="5" correct="true" />
            </comboBox>
         </shortAnswerQuestion>
         <!--
         combobox: The list of answers the user can select from
         value: The text of the choices
         correct: true for the correct choice in the list.
         -->

         <!-- This slide contains a short answer question with a rating -->
         <shortAnswerQuestion id="165" name="What is 2 1"
            points="1" timeLimit="0" attempts="3" survey="false"
            question="What is 2 + 1">
            <rating id="167" maximum="5" correct="3" />
         </shortAnswerQuestion>

         <!-- This slide contains a multiple choice question single answer -->
         <multipleChoiceQuestion id="18" name="What is 1 2" points="1"
            timeLimit="0" attempts="3" survey="false" question="What is 1 + 2">
            <radioButton id="20" correct="true" value="3" />
            <radioButton id="22" correct="false" value="4" />
            <radioButton id="24" correct="false" value="5" />
         </multipleChoiceQuestion>

         <!-- This slide contains a multiple choice question multiple answers -->
         <multipleChoiceQuestion id="42" name="What numbers are greater than 5"
           points="1" timeLimit="0" attempts="3" survey="false"
           question="What numbers are greater than 5">
            <checkBox id="44" correct="false" value="4" />
            <checkBox id="46" correct="true" value="6" />
            <checkBox id="48" correct="true" value="7" />
         </multipleChoiceQuestion>

         <!-- This slide contains a multiple choice question fill in the blanks -->
         <multipleChoiceQuestion id="54" name="Calculate the equations"
            points="1" timeLimit="0" attempts="3" survey="false"
            question="Calculate the equations">
            <textField id="56" caseSensitive="false" value="1 + 2">
               <answer value="3" />
            </textField>
            <textField id="58" caseSensitive="false" value="2 + 3">
               <answer value="5" />
            </textField>
            <textField id="60" caseSensitive="false" value="4 + 5">
               <answer value="9" />
            </textField>
         </multipleChoiceQuestion>

         <!-- This slide contains a multiple choice question select from lists -->
         <multipleChoiceQuestion id="66" name="Calculate" points="1"
            timeLimit="0" attempts="3" survey="false" question="Calculate">
            <comboBox id="68" value="3 * 3">
               <answer id="1" value="6" correct="false" />
               <answer id="2" value="9" correct="true" />
               <answer id="3" value="18" correct="false" />
            </comboBox>
            <comboBox id="70" value="Greater than 3">
               <answer id="1" value="2" correct="false" />
               <answer id="2" value="5" correct="true" />
               <answer id="3" value="6" correct="true" />
            </comboBox>
            <comboBox id="72" value="5 * 5">
               <answer id="1" value="25" correct="true" />
               <answer id="2" value="50" correct="false" />
               <answer id="3" value="10" correct="false" />
            </comboBox>
         </multipleChoiceQuestion>

         <!-- This slide contains a multiple choice question ratings -->
         <multipleChoiceQuestion id="79" name="Calculate the math" points="1"
           timeLimit="0" attempts="3" survey="false"
           question="Calculate the math">
            <rating id="81" maximum="5" correct="2" value="1 + 1" />
            <rating id="83" maximum="5" correct="3" value="1 + 2" />
            <rating id="85" maximum="5" correct="3" value="5 - 2" />
         </multipleChoiceQuestion>

         <!-- This slide contains a likert question -->
         <likertQuestion id="183" name="Answer the math" points="1"
           timeLimit="0" attempts="3" survey="false" question="Answer the math">
            <column id="1" value="Strongly Disagree" />
            <column id="2" value="Disagree" />
            <column id="3" value="Undecided" />
            <column id="4" value="Agree" />
            <column id="5" value="Strongly Agree" />
            <row id="1" value="1 + 1 = 2">
               <radioButton id="191" correct="false" column="1" />
               <radioButton id="192" correct="false" column="2" />
               <radioButton id="193" correct="false" column="3" />
               <radioButton id="194" correct="true" column="4" />
               <radioButton id="195" correct="false" column="5" />
            </row>
            <row id="2" value="2 + 2 = 3">
               <radioButton id="197" correct="false" column="1" />
               <radioButton id="198" correct="true" column="2" />
               <radioButton id="199" correct="false" column="3" />
               <radioButton id="200" correct="false" column="4" />
               <radioButton id="201" correct="false" column="5" />
            </row>
            <row id="3" value="1 + 2 = 3">
               <radioButton id="203" correct="false" column="1" />
               <radioButton id="204" correct="false" column="2" />
               <radioButton id="205" correct="false" column="3" />
               <radioButton id="206" correct="true" column="4" />
               <radioButton id="207" correct="false" column="5" />
            </row>
         </likertQuestion>

         <!-- This slide contains a drag and drop question -->
         <dragAndDropQuestion id="241" name="Match" points="1" timeLimit="0"
            attempts="3" survey="false" question="Match">
            <dragChoice id="243" name="5 + 5" type="message" />
            <dragChoice id="247" name="10 + 10" type="message" />
            <dragChoice id="251" name="3 * 10" type="message" />
            <dropArea id="245" name="Area 1" pairedWith="243" />
            <dropArea id="249" name="Area 2" pairedWith="247" />
            <dropArea id="253" name="Area 3" pairedWith="251" />
         </dragAndDropQuestion>

         <!-- This slide contains a click map question -->
         <clickMapQuestion id="262" name="Pick the Apple" points="1"
           timeLimit="0" attempts="3" survey="false" question="Pick the Apple">
            <clickArea id="264" correct="true" name="Area 1" />
            <clickArea id="266" correct="false" name="Area 2" />
            <clickArea id="268" correct="false" name="Area 3" />
         </clickMapQuestion>

         <!-- Here is a slide containing a pause zone -->
         <pauseZone id="144" name="Slide 30" points="0" timeLimit="0"
           attempts="0" survey="false" duration="34" />
         <!--
         The points, timeLimit, attempts, survey and duration attributes
         are irrelevant
         -->

      </content>
   </begin>
</report>


If all of the question details can't fit in a single 64KB block, a new attribute is added to the <content> tag "part=1:2" where the first number represents the chunk number and the second represents the total number of chunks.

For example:

    <content manifestVersion="3" part="1:2">

then

    <content manifestVersion="3" part="2:2">


After the begin tag, reports come for each question answered. They look like this.

After the begin tag, reports come for each question answered. They look like this for a slide that has just been completed.

<?xml version="1.0" encoding="UTF-8"?>
<report timestamp="1701" reportVersion="2" request="4281786798">
  <slide time="985" id="146" />
</report>


The last report has an additional <end/> tag to notify that the viewlet has ended. It also provides the total time, completed and a flag if they timed out.

<?xml version="1.0" encoding="UTF-8"?>
<report timestamp="201628" reportVersion="2" request="4281786798">
  <slide time="995" id="145" />
  <end timedOut="false" completed="true" time="200922" passed="true"
passingGrade="65" grade="85"/>
</report>


Sample Code

Here is some sample Java Servlet code that will collect the XML reports from a Viewlet into an array of XML documents.

import java.io.BufferedReader;
import java.io.IOException;
import java.io.PrintWriter;
import java.io.StringReader;
import java.util.ArrayList;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import javax.servlet.http.HttpSession;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.jdom.Document;
import org.jdom.input.SAXBuilder;

import com.qarbon.bristol.util.QStringUtilities;

public class ViewletStatsServlet extends HttpServlet
{
    private final Log log = LogFactory.getLog(ViewletStatsServlet.class);
   
    protected void doPost(HttpServletRequest request, HttpServletResponse response)throws ServletException, IOException
    {
        // Read in the message sent from the client.
        // This is an XML file submitted from a Viewlet.
        // Need to do the parsing here, rather than calling request.getParameters
        // because we want to read the stream as a single blob of text (for xml parsing)

        // Get all the data sent. This is a blob of XML posted from a Viewlet
        BufferedReader in = new BufferedReader(request.getReader());
        StringBuffer buf = new StringBuffer();
        String line = in.readLine();
        while(line != null)
        {
            buf.append(line);
            buf.append("\n");
            line = in.readLine();
        }
       
        String xml = buf.toString();

        Document report;
       
        try
        {
            StringReader xin = new StringReader(xml);
            SAXBuilder builder = new SAXBuilder();
            report = builder.build(xin);
        }
        catch (Exception e)
        {
            log.error("Error Processing Report: ", e);
            return;
        }

        boolean firstReport = report.getRootElement().getChild("begin") != null;
        boolean lastReport = report.getRootElement().getChild("end") != null;

        HttpSession session = request.getSession();
       
        if(firstReport)
            session.removeAttribute("reportCount");
       
        // Stuff the report into a session variable until the end report is reached.
        int reportCount = QStringUtilities.parseInt((String) session.getAttribute("reportCount"), 0);
        session.setAttribute("report" + reportCount, report);
        reportCount++;
        session.setAttribute("reportCount", String.valueOf(reportCount));

        if(lastReport)
        {
            // Gather all the reports
            ArrayList<Document> reports = new ArrayList<Document>();
            for (int i = 0; i < reportCount; i++)
                reports.add((Document) session.getAttribute("report" + i));
   
            // Clear the session
            if(!session.isNew())
                session.invalidate();
   
            // Process the reports
            processReports(reports);
        }
   
        // Return response
        response.setContentType("text/html");
        PrintWriter out = response.getWriter();
        out.println("<html><body><head>");
        out.println("<title>");
        out.println("Processed Report");
        out.println("</title>");
        out.println("</head>");
        out.println("<body><pre>");
        out.println("Processed Report");
        out.println("<pre></body></html>");
    }

    /**
     * This methods job is to tally up all the xml blobs sent from the user.
     * @param reports
     */
    private void processReports(ArrayList<Document> reports)
    {
        // TODO:
    }
   
}


Tags:
Home: User Guide What's new: Recently changed articles