Firewalls and Security in Software as a Service

One of the interesting outcomes of my recent course - Collaborative Learning Using Web 2.0 Tools - A Summary - was general consensus around:

1. Software as a Service is Great for Learning Professionals inside Corporations
   
2. Firewall restrictions still pose a problem for SOME services
   
3. Security is a concern, but generally should not stop use

The reason that Software as a Service is so attractive is that it is often hard to get Corporate IT to spend time on getting even simple software packages set-up and even harder to get them to agree to support these packages. Thus, while we are excited about wikis, blogs, discussion groups, etc., the practical reality is that, unless they already exist somewhere and we can piggy-back on those implementations, we are not going to be able to get them implemented by Corporate IT. Thus, there is real attraction in being able to sign up for hosted services that provide these tools without Corporate IT being involved.

For us to be successful doing this, we first need to make sure that the system will work with whatever firewall restrictions exist. For example, in our course, we found that Yahoo Groups were restricted in some corporate environments. Elluminate did not work through several firewalls, so we had to switch to WebEx. The Yahoo Toolbar (for MyWeb) couldn’t be installed on locked desktops. Instead, we should have used Del.icio.us. We had no trouble with our PBWiki. The good news is that there are lots of these services in most categories, and thus, the best advice is:

Test any service you are thinking of using in different locations and desktops to make sure that you are able to use the service effectively given firewall restrictions.

Do not believe any vendor claim that "it works through firewalls" because a firewall can be configured to stop anything it wants. That's its job.

The other big hurdle is the question of security. What's your exposure by having your content at a hosted location. The first part of the answer is whether outside parties (not you or the host) can hack into the system and get at your content. Generally, I think you will find that hosts provide fairly reasonable control, but you will want to check into their security approach.

The second part is that there is some set of administrators who provide the hosting who will have the ability to get in and see your content. The host may make it difficult for the administrator to get in there, but often its not that difficult. Really, this is the same situation as what you face in internal software with some set of Corporate IT staff having access to content (likely including email). In the case of hosted solutions, the added "risk" is that the administrators are employees of an outside company. On the other hand, you probably have better recourse against the host provider if the administrator does something wrong than you would against your own employee.

The security issue not new. There are likely lots of content types that get stored externally by your oganization. They might be using Salesforce.com as a CRM. They might be using an email system that handles Spam filtering and archiving. Chances are, the content you are putting up in your learning solution is far less of a risk than what is already getting stored out there. Which brings us to the first defense ... while the risk is probably low that you will actually have information leak out:

Try to limit content to information that would cause little damage to the
company if it were made public.

What if you need to work with content that is confidential and would potentially represent a risk? Well then you are going to need to go through the same protocols you would use internally to vet the system and likely you will again need to involve your IT staff because they are likely the ones who make these determinations. This will slow down your implementation time, but is not nearly the hurdle you have trying to bring software in-house.

Will they derail your process? If you look out at what's happening, you find mixed reviews. In an eWeek article: Security May Dog Software as a Service they provide a mixed answer:

the biggest challenge for companies such as Microsoft that see their future in on-demand software may be getting customers to understand and be comfortable
with the model.

And, the current state of network and application security at most companies is poor enough to make it hard to imagine on-demand deployments being any worse, experts agree.

You are still on the hook according to Software as a Service and Security:

A company must show due diligence in its relationships with third-party providers to ensure that those providers maintain and comply with U.S. and international regulations to which that company is subject. Under such regulations, it is the responsibility of the company—not the software as a service provider—to protect sensitive information.

The advice from an article in CFO Magazine:

Data security. Although SaaS vendors invariably emphasize the resources they devote to security, many customers remain uncomfortable with their employee and customer data flying over the Internet, not to mention potentially residing on the same data-center server as their rivals'. "Look at security. Do the due diligence. Make sure the vendor has the right premises and that protecting your data is its top concern," counsels David Brooks, director of CRM at Magma Design. Juniper Networks CIO Kim Perdikou insists on modifying SaaS contracts so that she has the right to do periodic security audits.

What's the bottom line? Chances are that you are not going to run into much of an issue. Try to keep the content to things where there is low risk. And where you have sensitive data, bring in IT staff to audit the security, bless the vendor(s), and check the protection in the contracts. It's still better than having to install software behind the firewall.

In talking with a lot of different CTOs from software development companies in Southern California it appears this is the way forward.

Keywords: eLearning 2.0, Web 2.0

Elves, Measuring Results and Informal Learning

Brent and I have been having a nice blog discussion. Our previous posts discuss what should be measured: Intermediate Factors in Learning, Intermediate Factors - Impact Many Measure One. And we finally seem to be agreeing with one exception. And this exception relates closely to my earlier concern eLearning Technology: Informal Learning is Too Important to Leave to Chance.

This discussion makes me think back to a question that I used to ask in my Computer Science Project class (based on something I read, but I now forget who it was - Fred Brooks maybe):

If an elf appeared and offer to give you a program that met your spec, how happy would you be?

After the initial jubilation wears off, the class realizes that there is some real concern on whether the program actually works as intended and without having insight into the guts of what's going on, it feels very uncomfortable. How do I know it works? How do I know what it's limitations might be?

It turns out that you really want more than just a program. You want one that you know how and why it works.

So, back to learning and measuring results. I actually don't want "just the outputs." I want to know how and why it's working. In the case of improving customer satisfaction based on knowledge of Store Layout and Product Knowledge, I want to know whether we've increased their knowledge and whether customer satisfaction has improved. While my client only cares about customer satisfaction, if it remains the same while knowledge increases, this is an important data point. It tells us about the system.

So, on to informal learning. Brent said in From Product Focus to Audience Focus:

The process is continuous and if our “training solution” is organic, dynamic, and flexible, it is very difficult to measure using the current method of measuring learning products. My point is “who cares”. If we have set up environments that help people collaborate, and support their informal learning, we should see output improvements.

And this is our only point of remaining disagreement. I would be much more comfortable if you can explain the internals of this system, how you know it works, when it will work and when it won't. As I said in Intermediate Factors in Learning ...

If you create an "organic, dynamic, flexible" learning solution but can't explain how it impacts the end numbers, then: (a) you won't get credit, (b) you won't know if you can repeat it successfully, and (c) you won't know if its really working.

Keywords: eLearning Trends, Informal Learning

Intermediate Factors in Learning

In Measure Intermediate or Final Factors, Brent responded to my posting: Technology: Intermediate Factors in Learning.

Brent and I (and Jay Cross and lots of others) agree that measures of butts in seats, number of completions, etc. are generally not that useful in telling what impact we are having on what matters to the business.

Where Brent and I really seem to disagree, and I've seen this other places is the importance of Intermediate Factors.

The key to almost every engagement for me is understanding how human performance drives the business. Yes, the client hires me to improve business results which is always ultimately around Revenue or Cost. Most of the time the client already measures intermediate factors such as Customer Satisfaction, Loyalty, Quality, etc. All of these are known to have impact on Revenue and Cost.

Most of my clients have some understanding of how human performance impacts these measures as well, but most often they have not fleshed this out to the level that is needed. Thus, if they tell me that they care about Revenue and they really want to look at improving Customer Satisfaction because that is the biggest predictor of Revenue. Then, I will drill down to what impacts Customer Satisfaction. Some aspects I won't be able to affect, but other aspects are within our ability to influence. So, we will continue down the path.

If they are a more sophisticated organization and have Customer Satisfaction Surveys, then likely we'll already have some very interesting intermediate factors defined. In home improvement retail, associates ability to tell customers where particular products are and getting them to the product is a big factor in customer satisfaction. Together with my client I will now agree that what I'm really trying to do is to impact these further intermediate measures such as the results of the Cust Sat Surveys around Associate Help in Finding Products. Likely I will further drill down on this to break it into Knowledge and Skill Components such as (a) Knowledge of Store Layout, (b) Knowledge of Product Categories/Types, (c) Handling of Customer Questions around Product Location.

I then do the most important thing with my client. I get agreement that what they are hiring me for is not to increase Revenue (end Factor), but rather to provide mechanisms that will impact these Knowledge and Skill Components in a way that improves scores on the particular Question (all are intermediate factors). In fact, when I work with the store managers as part of the intervention, I will make sure that they understand the relationships here and understand why these intermediate factors are important. Ultimately, the Causal Relationship that we believe will result in:

• If we improve Knowledge of Store Layout, Product Categories/Types, and How to Handle Customer Questions,
  
• we will improve the customer experience (as exhibit in how they rate us on that questions),
  
• which will improve customer satistfaction,
  
• which will improve revenue.
  

Much of what we do in Learning is making sure we understand these Intermediate Factors.

Keywords: eLearning Trends, Informal Learning

Tracking Without an LMS

Based on an earlier post - Tools for On-Demand Information - An LMS?, I received a couple of questions around tracking.

Then today, I saw a post on TrDev about tracking without an LMS and thought I should maybe clarify what I often see as the choices around tracking:

a. Click tracking
b. Custom tracking
c. LMS tracking

Click Tracking

In Click Tracking, you rely on looking at logs of what pages have been clicked on and get reports via log file analysis (web analytics) tools such as WebTrends. These tools will tell you:

• How many users have visited each page (HTML page)
• When users are visiting
• How long users stay

This is very standard technology that likely your IT shop can provide for you. If they cannot, then you can do what I've done on this blog and embed SiteMeter onto all of your web pages and it will give you similar kinds of reports. In fact, if you go to the link at the bottom of my blog (you can't see this in the RSS feed), you can see what traffic I get.

What you don't get with Click Tracking (without using some tricks) is the ability to see what any individual user did on the system. Thus, you couldn't tell if John or Sue finished the course. So, you have to answer the question:

Do I need to know if people are completing the course?

If the answer is no, then the other aspect to this solution is to create your course in a way that is easily tracked. Remember that Click Tracking only tells you what page was clicked on. This means that you need separate pages for your course. If you create a single, big Captivate Flash file, you will have no clickstream data. Instead you need to break the Captivate movies up (which is good practice anyhow) and put separate Captivate movies on each page.

LMS Tracking

I'm skipping Custom for a second. LMS tracking relies on creating a SCORM or AICC course which then communicates with the LMS in order to provide details of score, sections completed, which user it is, etc.

There are two issues with LMS tracking. First, many people do not have an LMS available to them. Second, even if you have an LMS you may not want to require users to login before they access content. This is discussed in Tools for On-Demand Information - An LMS?

Custom Tracking

While it is becoming less common as prices for LMS products have gone down and there are more hosted LMS products available, there are still times when we build custom tracking solutions. If you have no IT support available to you (i.e., simple programming), then this option is not available. However, there are some very simple things you can do to quickly and easily track your courses. While there are many solutions and lots of possible permutations, the basic approaches that are used are either a Simple Database or Enhanced Click Stream.

In a Simple Database approach, users will be asked to enter their name (sometimes at the start and sometimes at the end) in a simple form that will be recorded in a database. A simple web page is created that dumps out these results. There are lots and lots of subtleties here, but this is very simple to pull together and will give you a record of what a specific individual did. This approach is good when you only need a simple report of who has completed the content and do not need details of how they got there.

In Enhanced Clickstream, we will continue to rely on a tool like WebTrends, but we will put in place a simple bit of code that will enhance the clickstream data (the web log file) with information about the particular user. Normally, we rely on asking the user up-front for who they are (and then embed a cookie for repeat visits). This way, we can encode each page hit in the log file with the user information. WebTrends and other such tools can look at these parameters and give details of what pages that user has gone to. If you have a single completion page, it is easy to get a report on who has "completed" the course. This approach is good when you want to get more details of what individuals are doing on the system.

Keywords: eLearning Resources

Collaborative Learning Using Web 2.0 Tools - A Summary

Background

Over the past six weeks, I’ve been leading a course:

Collaborative Learning Using Web 2.0 / eLearning 2.0 Approaches

Course Description: The purpose of this course is to give you an opportunity to learn about collaborative learning by participating in collaborative learning. This course is designed to teach how to design and build collaborative learning experiences using Web 2.0 / eLearning 2.0 approaches.

You can find out more about the course itself from the Wiki at: http://collaborativelearning.pbwiki.com/

The basic structure of the course was:

• Six weeks long, each week had a one-hour virtual classroom session
• First two weeks were introductions to the tools and to collaborative learning
• Middle three weeks were the design (as a team) of collaborative learning projects, facilitation/participation in projects.
• Final week was a summary discussion

The participants in the class were corporate learning professionals from a variety of medium to large organizations.

Summary

This has been a great learning experience for me. I thought it might be interesting to provide some of the feedback from course participants and some of the insights from having conducted this course. I’m going to likely have additional posts based on the outcomes of this course. (Note: all quotes below are from participants.)

• Nuvvo was initially going to be used for the course as a registration system, communications and some content presentation. However, I found little value over using separate tools such as Yahoo Groups and a PBWiki. Through Nuvvo I did receive several registration requests, so it at least offers free advertising.
  
• There is significant interest in the topic. I sent out about 10 emails to friends and colleagues and quickly found about 20 people wanting to take the course (I got back 2 or 3 people at several companies). However, the low barrier to entry (free course offered by a friend) created an opportunity for a disconnect between my expectations for participants and their “commitment.”

“In order for the class to be effective I would be charging up front. Make some accountability for the folks to ensure the committment is there.”

• An introductory survey, conduct using SurveyMonkey – , worked very well! I would highly recommend SurveyMonkey. Using the survey I was quickly able to determine interest areas, when people would be able to participate, level of understanding of different technologies (Survey Results). My only problem was that I didn’t really have a way to change the class design significantly based on the result and I should have. See the next topic. Interestingly, one of the projects within the class (created by the participants) used a survey as well (which was a surprise, but speaks to how easy surveys are to use these days).

“Use a more comprehensive survey to learn more about the participants and why
they are taking the course and how they plan on using the things they learn. This might provide more insight in developing various exercises or homework - pairing people with like needs.”

• When I originally conceived the course, I assumed that most attendees would know about Blogs, Wikis, etc. generally, but would not have experience using them. This turned out not to be the case. Because of this I could easily have used another two weeks to more gradually introduce the tools.

"introducing 3 new technologies (blogs, BlogLines, wikis) at the same time is a bit much"
“Specific introductory directions would have been helpful.”
"liked to have started the week with more background and knowledge on blogs and wikis ... so my focus could have been on richer information sharing"
“Have a pre-class experience for those unfamiliar with the tools to learn more about
them by having demonstrations or showing examples. Or dedicate the first 2 sessions to learning and applying the tools one at a time in our homework. In this way, those people who are already familiar with them could be given the choice of missing those sessions. "

• All professionals (including learning professionals) are extremely busy. Even though the attendees in this class were fairly dedicated, it still is hard to find time for 3-5 hours per week. Ideally the course would have required less research time on topics. It is hard to define “assignments” that take relatively short amounts of time yet are interesting and deep enough. Part of this is the natural reluctance to share partial thinking.

"Doing research on the web is time consuming"
"I find that I am hesitant to write anything unless I have really thought it through."

• Firewalls and restrictive corporate environments caused us considerable grief: Yahoo Groups were restricted in some corporate environments, Elluminate did not work through several firewalls, we had to switch to WebEx and that really goofed up the start of the course; Yahoo Toolbar (for MyWeb) couldn’t be installed on locked desktops.
• Software as a Service (SaaS) is only way this could have worked. While I came into this believing in SaaS, I’m leaving it believing even more.


• Limit class size … teams of 4 worked well … 22 individuals acting individually in the early part of the course did not.


• I think that we achieved a different kind of understanding around Blogs, Wikis, Discussion Groups, etc. by actively using them as part of the class. Most of the participants gained real value from actively working with the tools as part of a learning experience, BUT, there is definite frustration as well given having to simultaneously get up to speed on the tools and learn about using them. Given the widely different levels of experience and propensity for using the tools, it was very difficult to balance learning about the tools and using them as part of learning. In the future, I will separate the two entirely.

"Let’s hear it for failure based learning"

• Students were given a first-hand experience relative to "control" in collaborative learning both as participants and as leaders. As participants, they were given a lot of autonomy which sometimes worked well and many times did not. As leaders of their individual projects, they had mixed results with the participants actively engaging on the project. Also, I was fairly open on many issues such as assigning roles in teams, establishing norms – but some teams suffer because of this:

"Team collaboration might have been better if we had assigned roles soon after Tuesday's meeting"
"I found that in our collaborative environment, it was much easier to get busy and not participate as much as I would have when forced to face my facilitator or team members in an actual conference call or live meeting encounter. It became apparent that deadlines would need to be set and enforced in a corporate collaborative environment to ensure things kept progressing."

• You still need to enforce timelines, norms, etc. While it’s nice to try to leave things open, it may be more effective to be somewhat dictatorial.

“Meetings are great to keep things moving because they are a deadline”

• Each week, students would reflect on what they learned and how the class was working for them through a Plus/Delta assignment. This worked very well and sparked interesting discussions each week. But you need a thick skin. Most of the quotes here comes from the Plus/Deltas. However, there was some question of the format:

"I am not sold on the value of a plus / delta post. I prefer a more free form comment on the learning of the week. The important element of the post from a learning perspective is learner reflection. I think general comments on the week
allows this reflection to be more meaningful, at least to me it is."

• The tools generally work pretty well for collaboration

"Our new collaboration tools were essential as our team had challenges with time schedules and difficulties coming together."
"signing up on Yahoo 360 and Yahoo My Web went well"
"sharing links with My Web is a lot easier than e-mailing them"
"Yahoo Bookmarks allows you to see your bookmarks from home or office"
"Yahoo groups worked well for exchanging information with team members"
"Signing up for Yahoo 360 and My Web were very quick and easy"
"I was surprised at how easy add-ins were to incorporate"

• The projects in the course (designed by participants) was a great learning experience. If anything, more of this kind of learning would have been better.


• Timing is a real challenge. Ideally you would get work into pairs or teams quickly, but with this kind of class you expect to have some level of non-participation and you don’t want people teamed with folks who will drop. So, you start individually, but then move to more team-based. However, most of the participants felt that pairs/teams were more effective.


• Our teams had mixed results in really collaborating virtually. This was primarily a function of timing. If you could do several cycles in a short amount of time, then you could get good collaboration.

Keywords: eLearning 2.0, Web 2.0, Collaborative Learning

What does Google Trends tell us?

Google Trends is quite interesting to play around with to see how different search terms have been used within Google. For example, I looked at eLearning as compared with Knowledge Management...




You can see that eLearning and Knowledge Management have both been trending downward in searches. Does that imply something as is suggested in: Organic KM: KM Is Losing Steam - Evidence From "The Database Of Intentions" ?

I'm not so sure that you can directly jump to the conclusion that interest in these topics are waning. Rather, I think with both Knowledge Management and eLearning the conversations have shifted as they've become more day-to-day kinds of concerns. On the other hand, I do think you can glean interest in other topics such as:



Which to me seems like a fairly accurate depiction of the interest that I've heard at conferences over the last few years.

Keywords: eLearning Trends

Fun Headline Generator

This is a fun headline generator tool: Headline Generator http://tools.fodey.com/generators/newspaper/snippet.asp

You give it a date, and the headline to generate, the news story and it generates the newspaper for you. A fun little thing to do in your eLearning course. So go generate headlines!

Update - here's another type of generator. The Sign Generator.
Keywords: eLearning Resources