Integrating ESOE with Blackboard

Authors
Bradley Beddoes
Craig Comerford

Applicable versions (ESOE)
Beta 2, 0.51+

Applicable versions (Blackboard)
7.1+ 7.2+ 8.x to be testing Sept 08. NG will be configured and tested as soon as it is available.

Overview

Blackboard is an online learning and teaching environment in use at many large education institutions today. We have worked with the Blackboard API to integrate ESOE into Blackboard to handle all user authentication.

Installation

Blackboard installation builds from a standard Java SPEP install. You should first be familiar with the document Java SPEP Installation Guide and comfortable with a generic SPEP install. For blackboard locations and steps vary slightly from a standard install so we'll follow the guidelines below.

This document assumes that $blackboard is the home directory of your blackboard installation. e.g: /usr/local/blackboard

You will need to download the SPEP and Blackboard integrator binaries to undertake this work from Downloads and have them available on your host before continuing.

spepinteg-blackboard.tar.gz (contains spepinteg-blackboard.jar), spep.tar.gz (contains spep-endorsed.tar.gz, spep.config and spep.war), spepfilter.tar.gz.

Preparing your server

To successfully operate the Blackboard SPEP integration your server must be accessible on port 443 from the ESOE system. You can ask your ESOE system administrator for the IP addresses ESOE will connect from if your running a restrictive firewall policy and update it appropriately. In some circumstances your administrator may allow you to configure port 80 unsecured SPEP instances. This is a per installation decision which your ESOE system administrator can advise you of.

The procedure for installing binaries is as follows (ensure permissions are set correctly for your blackboard user):
  1. Backup jar files in $blackboard/apps/tomcat/common/endorsed. For Version 9 (Tomcat 6) the endorsed directory by default is no longer used so create a new directory "endorsed" $blackboard/apps/tomcat so you will copy the jars to $blackboard/apps/tomcat/endorsed. Point 13 relates to this. For further reference see http://tomcat.apache.org/tomcat-6.0-doc/class-loader-howto.html and http://java.sun.com/javase/6/docs/technotes/guides/standards/index.html.
  2. Extract spep-endorsed.tar.gz to $blackboard/apps/tomcat/common/endorsed
  3. Extract spep.tar.gz to a local working directory
  4. Create a directory called spep in your local working directory and extract the spep.war file inside this directory (this will look like the spep directory for a normal tomcat webapps deployment of spep.war)
  5. Copy the spep directory to $blackboard/webapps/spep
  6. Create a core directory to store spep artifacts, a sub directory called config to store configuration and a sub directory called logging for log files. This must be readable by the user blackboard runs as. e.g /opt/spep/config and /opt/spep/logging
  7. Copy the spep.config file to the configuration directory e.g /opt/spep/config/spep.config
  8. At this point you have a choice in how to advise the SPEP install of your configuration location
    1. Java Property (RECOMMENDED). Export a java property called spep.data to your Java environment when blackboard starts. This should be set to the top level artifacts directory. e.g -Dspep.data=/opt/spep
      To do this correctly for blackboard edit the file $blackboard/apps/tomcat/bin/blackboard-tomcat.sh.bb, look for the settings of JAVA_OPTS, append the line:
      JAVA_OPTS="$JAVA_OPTS -Dspep.data=/opt/spep"
    2. Create a Java properties file called spepvars.config and place it in $blackboard/webapps/spep/WEB-INF/ this is a standard java properties file format, set the variable spep.data to the same value you would above. We don't recommend this method though it may be useful in a multi spep environment, be sure to save your spepvars.config file safely, it will need to be copied back to the web application on each upgrade you perform. This seems to be the preferred option for Blackboard V9.
  9. Extract the file spepfilter.tar.gz to your local working directory and copy the resulting jar files to $blackboard/webapps/login/WEB-INF/lib/ AND $blackboard/webapps/portal/WEB-INF/lib
  10. Extract the file spepinteg-blackboard.tar.gz to your local working directory and copy spepinteg-blackboard.jar:
    $blackboard/systemlib
  11. Navigate to the directory $blackboard/config/tomcat/classpath
  12. Create a file called 'spep-common.classpath.bb'
  13. Edit $blackboard/config/tomcat/conf/wrapper.conf.bb and add the line wrapper.java.additional.20=-Djava.endorsed.dirs=@bbconfig.basedir@/apps/tomcat/endorsed (make sure additional.20 isn't already taken otherwise increment the number).
  14. In this file add a single line of:
    $blackboard/systemlib/spepinteg-blackboard.jar
    YOU MUST MANUALLY EXTRACT $blackboard e.g.: /usr/local/blackboard/systemlib/spepinteg-blackboard.jar
  15. Open $blackboard/system/build/bin/launch-tool. sh | bat.
    Add the lines BB_CP=$BB_CP:spepinteg-blackboard.jar and BB_CP=$BB_CP:log4j.jar under other BB_CP lines.
  16. Add a reference to spepinteg-blackboard.jar in the PERL file that calls the Java email sending routine - more specific details to come shortly (sept 08). (resolves an email send issue)
  17. Check to ensure all instances of email.jar have a corresponding activation.jar in the same directory. (resolves an email send issue) - mail.jar added to /usr/local/blackboard/apps/tomcat/common/endorsed/
  18. (probably do NOT need these - possible xercesImpl.jar and xml-apis.jar needed in $blackboard/systemlib/)

SPEP Registration

Navigate to the "Register Service" page of the ESOE Manager.

Enter values for the following fields:

Field Description
Service name Human readable name for the service
Service URL Base URL or "home" of the service - the main entry point for a user agent e.g. https://blackboard.company.com
Service Identifier Identifier of the service must be a URI, generally this is the same as the value entered above
Service description A description of what the service does
Service Authz Failure A message to be displayed by the central authorization failure page when access to one of your resources has been denied. This may contain HTML markup.

Click Next.

Add Service Contacts

Select the contact type from the list, and enter the other details for the contact.
When this is filled in, click "Save contact".

You may edit and delete contacts after they have been saved. After you have finished entering contacts, click Next.

Add Service Nodes

The "SPEPNode URL" is used as the base URL for the SPEP endpoints. If your running blackboard on multiple nodes in a Layer 7 load balanced environment you will need to enter each of them here (you do not need a node for the service URL). The node domain names may be unrelated to the service domain if this is how your setup. e.g. https://blackboard.company.com as a service address may have node URL's of https://somenode.company.com and https://someothernode.company.com

The different service locations here are paths to the service endpoints, using the SPEPNode URL as a base.

Select the server technology you are using. The default values for the endpoint paths will work for a default SPEP deployment. For Blackboard you need Java.

Click Save Node. You may add more nodes and edit or delete nodes that have already been added.

After you have finished entering nodes, click Next.

Finalize registration process

From this page you may check the information you have entered, and also navigate back to previous pages to make any alterations by clicking the Previous button.

Once you are happy with the configuration, click Complete.

Service configuration

Navigate to your service in the ESOE Manager application. Click on the "get service configuration" link. In the service description, click on the "service node configuration" link.

This page shows you the configuration values that need to be set up in the spep.config file, which can be found at the location you've earlier configured for spep.data.

For each of the configuration options shown by the ESOE Manager service node configuration page, enter the corresponding value for the current node in its spep.config file.

It is very important that each node's configuration be correct. There may be some differences between them.

Add the following line to spep.config under the definitions for logoutClearCookie.1
logoutClearCookie.2=session_id

BELOW the line (in the part it says not to edit anything), check the following:

# Time that sessions which have yet to complete an authentication event are considered valid
sessionCacheTimeout=3600 (default value was 120, should be 3600)

Save the keystore file by clicking the "download keystore" link. The keystore will be the same for all nodes. Store the keystore as spep.data/config/spepKeystore.ks

Blackboard configuration

Integrate with the core Blackboard session handler

Edit the file $blackboard/config/authentication.properties

At the very top of this file add the following:
########################################################
##
##    ESOE SPEP Authentication Properties
##
########################################################

auth.type.esoe-spep.impl=com.qut.middleware.spep.integrators.blackboard.BlackboardIntegrator
auth.type.esoe-spep.userID=uid
auth.type.esoe-spep.fullName=name
auth.type.esoe-spep.mail=mail
auth.type.esoe-spep.roles=roles
auth.type.esoe-spep.logoutURL=https://esoe-tst.qut.edu.au/web/logout.htm

N.B: The values for userID, fullName, mail, roles and logoutURL are specific to each sites installation and attribute release policies. Query your ESOE system administrator for the values which should be placed here. Edit the file $blackboard/config/bb-config.properties
Change the value of bbconfig.auth.type to esoe-spep
bbconfig.auth.type=esoe-spep

Integrate with the Login Webapp

To integrate with the webapp you must edit BOTH web.xml.template and web.xml

Edit the file $blackboard/webapps/login/WEB-INF/web.xml.template and change it to read as follows:
<?xml version="1.0" encoding="ISO-8859-1"?>

<web-app xmlns="http://java.sun.com/xml/ns/j2ee" 
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
         xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee web-app_2_4.xsd" 
         version="2.4">

  <display-name>LoginBrokerServlet</display-name>

  <filter>
        <filter-name>spep-filter</filter-name>
        <filter-class>com.qut.middleware.spep.filter.SPEPFilter</filter-class>

        <init-param>
                <param-name>spep-context</param-name>
                <param-value>/spep</param-value>
        </init-param>
  </filter>

  <servlet>
        <servlet-name>LoginBrokerServlet</servlet-name>
        <servlet-class>blackboard.platform.security.authentication.servlet.LoginBrokerServlet</servlet-class>
  </servlet>

  <filter-mapping>
        <filter-name>spep-filter</filter-name>
        <url-pattern>/*</url-pattern>
  </filter-mapping>

  @web-jsp-declarations@

  <servlet-mapping>
        <servlet-name>LoginBrokerServlet</servlet-name>
        <url-pattern>/</url-pattern>
  </servlet-mapping>

</web-app>
Likewise edit the $blackboard/webapps/login/WEB-INF/web.xml file and modify the header to read as follows (... indicates extra content already present in the file you MUST leave inline):
...
  <display-name>LoginBrokerServlet</display-name>

  <filter>
        <filter-name>spep-filter</filter-name>
        <filter-class>com.qut.middleware.spep.filter.SPEPFilter</filter-class>

        <init-param>
                <param-name>spep-context</param-name>
                <param-value>/spep</param-value>
        </init-param>
  </filter>

  <servlet>
        <servlet-name>LoginBrokerServlet</servlet-name>
        <servlet-class>blackboard.platform.security.authentication.servlet.LoginBrokerServlet</servlet-class>
  </servlet>

  <filter-mapping>
        <filter-name>spep-filter</filter-name>
        <url-pattern>/*</url-pattern>
  </filter-mapping>

<!--
Automatically created by Apache Jakarta Tomcat JspC.
Place this fragment in the web.xml before all icon, display-name,
description, distributable, and context-param elements.
-->
...

Integrate with the Portal Webapp

To integrate with the webapp you must edit BOTH web.xml.template and web.xml

Edit the file $blackboard/webapps/portal/WEB-INF/web.xml.template and change it to read as follows (... indicates extra content already present in the file you MUST leave inline):
<?xml version="1.0" encoding="ISO-8859-1"?>

<web-app xmlns="http://java.sun.com/xml/ns/j2ee" 
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
         xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee web-app_2_4.xsd" 
         version="2.4">

  <display-name>Portal</display-name>

        <filter>
                <filter-name>spep-filter</filter-name>
                <filter-class>com.qut.middleware.spep.filter.SPEPFilter</filter-class>

                <init-param>
                        <param-name>spep-context</param-name>
                        <param-value>/spep</param-value>
                </init-param>
         </filter>

        <servlet>
                <servlet-name>TabPortalServlet</servlet-name>
                <servlet-class>blackboard.portal.servlet.TabPortalServlet</servlet-class>
        </servlet>
....
   @web-jsp-declarations@

        <filter-mapping>
                <filter-name>spep-filter</filter-name>
                <url-pattern>/*</url-pattern>
        </filter-mapping>

    <servlet-mapping>
                <servlet-name>TabPortalServlet</servlet-name>
                <url-pattern>/myinst/*</url-pattern>
        </servlet-mapping>
...

</web-app>

Likewise edit the $blackboard/webapps/portal/WEB-INF/web.xml file and modify the header to read as follows (... indicates extra content already present in the file you MUST leave inline):
<?xml version="1.0" encoding="ISO-8859-1"?>

<web-app xmlns="http://java.sun.com/xml/ns/j2ee" 
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
         xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee web-app_2_4.xsd" 
         version="2.4">

  <display-name>Portal</display-name>

         <filter>
                <filter-name>spep-filter</filter-name>
                <filter-class>com.qut.middleware.spep.filter.SPEPFilter</filter-class>

                <init-param>
                        <param-name>spep-context</param-name>
                        <param-value>/spep</param-value>
                </init-param>
         </filter>

        <servlet>
                <servlet-name>TabPortalServlet</servlet-name>
                <servlet-class>blackboard.portal.servlet.TabPortalServlet</servlet-class>
        </servlet>
...
        <filter-mapping>
                <filter-name>spep-filter</filter-name>
                <url-pattern>/*</url-pattern>
        </filter-mapping>

    <servlet-mapping>
        <servlet-name>blackboard.web.caret_jsp</servlet-name>
        <url-pattern>/caret.jsp</url-pattern>
    </servlet-mapping>
...
</webapp>

Integrate with Blackboard Apache Instance

Edit the file $blackboard/apps/httpd/conf/httpd.conf.bb
Add the line:

JKMount       /spep/*                     @bbconfig.tomcat.main_worker@

after similar configuration lines in this file.

Integrate with the Blackboard Tomcat Instance

Edit the file $blackboard/config/tomcat/conf/server.xml.bb
Add the line:

<Context docBase="@@bbconfig.basedir@@/webapps/spep"       path="/spep"       crossContext="true"/>

after similar configuration lines in this file.

Integrate with Blackboard logging

Edit the file $blackboard/config/tomcat/common/classes/log4j.properties.bb to look like:
################################################################################
#
# Blackboard log4j logging options for Tomcat
#
# See http://logging.apache.org/log4j/docs/manual.html for more details.
#
################################################################################

log4j.rootLogger=INFO, R
log4j.appender.R=org.apache.log4j.DailyRollingFileAppender
log4j.appender.R.DatePattern='.'yyyy-MM-dd'.txt'
log4j.appender.R.File=@@bbconfig.basedir@@/logs/tomcat/${bbconfig.appserver.cluster.id}${cluster.divider}catalina-log.txt

log4j.appender.R.layout=org.apache.log4j.PatternLayout
log4j.appender.R.layout.ConversionPattern=%p %t %c - %m%n

#
# Uncomment the following three log4j lines to use size-based logfile rotation
#
#log4j.appender.R=org.apache.log4j.RollingFileAppender
#log4j.appender.R.MaxFileSize=10MB
#log4j.appender.R.MaxBackupIndex=10

log4j.appender.spep=org.apache.log4j.DailyRollingFileAppender
log4j.appender.spep.DatePattern='.'yyyy-MM-dd'.txt'
log4j.appender.spep.File=/opt/spep/logging/spep.log
log4j.appender.spep.layout=org.apache.log4j.PatternLayout
log4j.appender.spep.layout.ConversionPattern=%p %t %c - %m%n

log4j.appender.spepauthn=org.apache.log4j.DailyRollingFileAppender
log4j.appender.spepauthn.DatePattern='.'yyyy-MM-dd'.txt'
log4j.appender.spepauthn.File=/opt/spep/logging/spep-authn.log
log4j.appender.spepauthn.layout=org.apache.log4j.PatternLayout
log4j.appender.spepauthn.layout.ConversionPattern=%p %t %c - %m%n

log4j.appender.spepauthz=org.apache.log4j.DailyRollingFileAppender
log4j.appender.spepauthz.DatePattern='.'yyyy-MM-dd'.txt'
log4j.appender.spepauthz.File=/opt/spep/logging/spep-authz.log
log4j.appender.spepauthz.layout=org.apache.log4j.PatternLayout
log4j.appender.spepauthz.layout.ConversionPattern=%p %t %c - %m%n

#
# Use less verbose logging for the DWR package
#
log4j.category.uk.ltd.getahead.dwr.convert=WARN, R
log4j.category.uk.ltd.getahead.dwr.impl=WARN, R
log4j.category.uk.ltd.getahead.dwr.util=WARN, R

log4j.category.com.qut.middleware.spep=DEBUG, spep
log4j.category.spep.authn=DEBUG, spepauthn
log4j.category.spep.authz=DEBUG, spepauthz

NB: If your spep.data location isn't /opt/spep then you'll need to adjust this file accordingly

Deploy changes and restart Blackboard

The last step is simply to restart the Blackboard instance and push out our configuration changes by using the command PushConfigUpdate bin | sh. Don't forget to enable your service in ESOE Manager and ensure you have an authorization policy giving you access (everyone is denied access to new services by default).

Troubleshooting

If you encounter Server 500 errors, check that the JAVA_OPTS setting we added at step 8.1 has persisted after the PushConfigUpdate (in the file $blackboard/apps/tomcat/bin/blackboard-tomcat.sh.bb).

Also ensure if you are using a load balancer that you employ a suitable persistence method for the nodes in your pools. Do NOT use IP address as user traffic via HTTP proxies may come from a different IP address to their HTTPS traffic and break the session establishment. Cookie persistence (set by the load balancer, ie F5 BigIP) is known to work fine.

ALSO NOTE THAT A PUSHCONFIGUPDATE.SH POTENTIALLY messes with the $BLACKBOARD/apps/tomcat/conf/server.xml configuration. Check it esp if you have Server 500/javax.servlet.ServletException: Couldn't initialize SPEP/com.qut.middleware.spep.filter.exception.SPEPInitializationException: SPEP couldn't be initialized. No SPEP in this servlet context (yet?)type messages.

Feedback

We aim to continually improve this documentation set to make it as easy as possible for new users and seasoned users alike to setup Blackboard integration. We welcome any comments or additions you may have on the ESOE users mailing list at any time.