As an alternative to our provided online survey hosting services, Sawtooth Software also allows users to publish online surveys onto their own servers (with a cloud hosting partner, located on-premises, or in any other hosting solution). This hosting arrangement is what we refer to as “self-hosting” surveys.
Before you begin
Self-hosting is an offering intended for teams with prior experience managing web server and database systems. Sawtooth Software cannot set up a self-hosted system for you, and our technical support will provide a good faith effort to help troubleshoot issues you may come across, but we make no guarantees that the self-hosting system you set up will function in a specific way or provide any guaranteed level of performance. We are only able to provide a limited amount of hosting support to customers who choose to self-host surveys, as individual implementations of server and database systems can vary greatly. An alternative to self-hosting is renting a dedicated server hosted by Sawtooth Software and may better meet your survey hosting needs. Send an email to firstname.lastname@example.org for costs and details about dedicated server rentals.
Lifecycle of a self-hosted survey
A survey author creates a survey using Lighthouse Studio (a desktop application) on a Windows computer. When the survey is ready to field, Lighthouse Studio packages the survey so it can be hosted on a webserver. Lighthouse Studio creates one web application package per survey. This package includes executable Perl files that run the survey and store response data in a SQL database. Included in the web application package is an Admin Module that provides a password protected interface accessible via a web-browser for monitoring and reviewing live survey data. The generated package is self-contained and does not require authentication with Sawtooth Software to run.
A self-hosted survey can be uploaded to a given server using SFTP from within Lighthouse Studio, or the web application package can be manually moved to the web server using another method (a third-party FTP client, network connection, etc.)
After uploading the survey, the survey author will send out a survey URL to respondents and manage the survey using the web application’s included Admin Module. Lighthouse Studio cannot send out email invites, but surveys authored in it do support unique links generated by other mass email platforms. When the desired number of respondents have been collected, the survey author can close the survey using the Admin Module.
1. Web Server
- Linux or Windows host (OS should be relatively up-to-date with active vendor support for security patches and updates)
- Internet-facing software for web traffic
- NGINX (version 1.12 or later)
- Apache (version 2.3 or later)
- Microsoft Internet Information Services (IIS) (Windows Server 2012 or later)
- SQL-compatible database:
- MySQL (version 5.7 or later)
- MariaDB (version 10.2 or later)
- Microsoft SQL Server (2012 or later)
- Database for the survey(s) on the server
- Database user for the Perl survey application
- Database name and database user credentials must be set in Lighthouse Studio prior to the survey being packaged for hosting
- Perl Runtime Environment (Perl version 5.14.2 or later)
- Perl Modules:
- DBI - Database Interface Module (version 1.6 or later)
- DBD:: MySQL - MySQL Database Driver Module (version 4.0.2 or later)
- Only required if using a MySQL/MariaDB database
- DBD::ODBC - SQL Server Database Driver Module (minimum version will depend on version of SQL Server used)
- Only required if using a SQL server database
- JSON::PP - Pure Perl JSON Encoder/Decoder Module (version 4.00 or later)
- (Optional) DateTime - Date & Time Module (version 1.06 or later)
4. SFTP or FTP access to upload the survey files to the server
- The webserver must be able to execute the files that are uploaded (file ownership and permissions must be set properly)
- The webserver file root must be set to or linked to the uploaded files
NGINX is Sawtooth Software’s preferred web server. However, any web server that can execute scripts via the CGI protocol should be able to run Lighthouse Studio generated surveys.
We have seen the best stability and performance of Lighthouse Studio generated surveys on Linux servers using NGINX, Perl, FastCGI (fcgiwrap) and a MySQL Database. Our experience shows that this configuration uses the least RAM with the best throughput, especially while under heavy loads.
Other common configuration stacks include:
- Linux with Apache using mod_perl or mod_proxy to connect to Perl through FastCGI with MySQL as a database.
- Windows Server with Microsoft IIS, SQL Server, and StrawberryPerl.
- Windows servers will generally utilize more RAM when running surveys than Linux servers.
- StrawberryPerl is a relatively easy-to-use and stable Perl environment for Windows servers.
Regardless of the software stack, the web server ought to be configured to serve static files to the client while passing web requests for the perl scripts to the Perl runtime environment through the CGI protocol.
We recommend using the latest stable, generally available version of a SQL-compatible database (e.g. MySQL, MariaDB, or MS SQL Server).
If your self-hosting environment uses Microsoft Windows, it is possible to use either a SQL Server or MySQL database on the host.
If your self-hosting environment uses Linux, it may be possible to host Microsoft SQL Server for Linux, but configuration of Perl packages to connect the web application to the Linux-based SQL Server instance may be difficult.
Depending on your hosting needs, it may make sense to connect a survey web application to an already-existing database host/instance, run a local database on the same host as the web application, connect to a managed cloud database service, or set up a distinct database host just for survey data. All configurations are possible with Lighthouse Studio- authored surveys. Regardless of what host architecture looks like, we suggest setting up a specific database and user with at least SELECT, INSERT, UPDATE, CREATE, ALTER, DROP, DELETE, and INDEX privileges on the database. The database user credentials must be input into Lighthouse Studio before the Perl web application is packaged for upload. Make sure the database host is accessible from the Perl runtime environment on the web server. As part of the first upload of a survey web application package, the Perl scripts contained in it will connect to the database and set up anywhere from 8 to 100+ tables in the database, depending on the amount of data the survey author has programmed/built the survey to collect.
File Transfer (SFTP, FTP)
Lighthouse Studio has a built-in “auto-uploader” that utilizes FTP, SFTP, or FTPS connections to upload new surveys to the server (or to make updates to an existing survey by oveerwriting files that are already on the server). This means that servers configured to listen for these connections will allow survey authors to “deploy” surveys and updates using Lighthouse Studio without server administrator intervention. Some guidelines to consider when setting up the SFTP server:
- Most Linux distributions that accept SSH connections can be easily configured to also allow SFTP connections without the need to install additional software.
- Make sure the folder to which the SFTP user can upload is the same as (or linked to) the public web directory of the webserver.
- Make sure the webserver has execution permissions on the uploaded files.
- Lighthouse Studio requires a username and password to utilize FTP, SFTP, or FTPS (key-based authentication is not currently supported).
Lighthouse Studio’s “auto-uploader” will attempt to set correct permissions on all files it uploads. If files are uploaded in some other way, you may need to set file permissions according to this chart:
|Every folder in /graphics||707|
In order to help server administrators confirm that their configurations are correct, Sawtooth Software has published three Perl application test scripts that can be used to verify correct server functionality before a survey is uploaded. These files should be placed in the root directory of the webserver (where survey files are expected to reside) and should be given execute permissions:
- test1.pl - this script simply tests that Perl is executing properly. Navigating to this test script in your browser should output plan, rendered HTML if the server is configured properly.
- test2.pl - this script displays the working directories available in the Perl runtime environment, as well as the files in the current working directory for debugging purposes. Perhaps most importantly, this script also displays whether the recommended perl modules are installed and accessible in the Perl runtime environment.
- db_test.pl - this script, when modified in a text editor to include a database type, hostname, db name, and credentials, will verify that a database connection can be successfully made.
Configuring a self-hosted server with sufficient specifications (CPU types, number of cores, RAM, etc.) to handle survey traffic is a tricky yet crucial task. A robust server configuration depends considerably on survey traffic. We recommend working with survey authors to determine a survey fielding strategy that balances survey traffic with server power. A strategy that invites a high number of survey respondents at once will overwhelm smaller servers. Our recommendation is to plan server capacity for the highest expected traffic load, and to send survey invites in batches that will not exceed server resources.
Example Survey Traffic
In our experience, survey traffic generally surges soon after respondents are invited and then tapers off as time goes on. Because of this behavior we recommend spacing out survey invitations in smaller groups over time to avoid a high volume of concurrent respondents.
Overwhelming the server with survey traffic can impact survey performance and respondents may see error messages. This could potentially result in more incomplete responses than desired. A survey invitation plan is especially important if there are multiple surveys hosted on the same server.
Estimating Survey Traffic
The best estimate of survey traffic is the expected survey response rate, or the number of people estimated to answer the survey divided by the number of people invited to take the survey. For example, for an email list of 10,000, if 2,000 people are expected to respond, the response rate would be 20%.
When you know the estimated response rate and the number of invitations that may be sent out for a given survey, you can determine if your server configuration is robust enough to handle the traffic. If not, you may either need to invite smaller batches of respondents at a time or increase the server specifications to handle more concurrent respondent connections.
In some cases, a small portion of the email list can be tested on the server to better estimate what the response rate might be and gauge the server’s performance, indicating whether a more performant server is needed.
If you have any questions, please reach out to us at email@example.com
Disclaimer of Liability
The material and information contained on this page is for general information purposes only. You should not rely upon the material or information on this page as a basis for making any business, legal or any other decisions. Whilst we endeavor to keep the information up to date and correct, Sawtooth Software, Inc. makes no representations or warranties of any kind, express or implied about the completeness, accuracy, reliability, suitability or availability with respect to the page or the information, products, services or related graphics contained on the page for any purpose. Any reliance you place on such material is therefore strictly at your own risk.