Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
88 commits
Select commit Hold shift + click to select a range
c796c32
updated google analytics to new POST interface
Jan 7, 2015
2a8139e
removed mailinglist setup from existing account tab
Jan 20, 2015
86774c4
Merge pull request #664 from paczian/develop
Jan 20, 2015
0f326b8
empty metadata is {} and not null
teharrison Jan 20, 2015
42c2440
Merge pull request #665 from teharrison/develop
teharrison Jan 20, 2015
28b4645
added private option to project resource
Jan 23, 2015
aefa4b0
added shock preferences
Jan 23, 2015
5764eed
Merge branch 'develop' of github.com:MG-RAST/MG-RAST into develop
Jan 26, 2015
77b5135
enabled SHOCK preferences object
Jan 26, 2015
5cb8e51
removed shock auth from the user preferences until shock accepts mg-r…
Jan 26, 2015
dc44513
fixed quoting bug in auth
Jan 26, 2015
ba5139a
fix feature output of annotation resource
teharrison Jan 26, 2015
03e5dcc
Merge pull request #667 from teharrison/develop
teharrison Jan 26, 2015
862f759
fixed grouping calculation
Jan 27, 2015
0bb59fa
Merge pull request #669 from paczian/develop
teharrison Jan 28, 2015
9fc4f15
Bug fix
jaredbischof Jan 28, 2015
d802561
Merge pull request #671 from jaredbischof/develop
teharrison Jan 28, 2015
f1c51ea
added summary verbosity
Jan 29, 2015
e0ae1d4
fixed bug in summary verbosity
Jan 29, 2015
9c50e78
fixed another bug in summary verbosity
Jan 29, 2015
8556648
guess pinky...
Jan 29, 2015
43a41a6
fixed shock preferences
Feb 2, 2015
c5bd0d2
udpated to enable optional mg-rast auth
Feb 2, 2015
6534e51
fixed user preferences retrieval
Feb 3, 2015
502289e
Merge pull request #673 from paczian/develop
Feb 3, 2015
910a2db
Edited Shock ACL interaction to match new Shock api.
jaredbischof Feb 3, 2015
fd32d8b
Merge pull request #674 from jaredbischof/develop
teharrison Feb 3, 2015
6e4d537
add resubmit action, fix retrieval of awe id
teharrison Feb 4, 2015
e90946a
Merge pull request #676 from teharrison/develop
teharrison Feb 4, 2015
8ac6478
added heartbeat resource
Feb 9, 2015
ff6131c
Merge pull request #680 from paczian/develop
Feb 9, 2015
98334a8
fix function name
teharrison Feb 9, 2015
593d9dc
minor tweaks to how inbox nodes are found
Feb 12, 2015
b534363
fixed function name
Feb 12, 2015
31d64ac
fix job creation api
teharrison Feb 12, 2015
d621a2a
bug fixes to job submission
teharrison Feb 12, 2015
c20a36a
Merge pull request #681 from teharrison/develop
teharrison Feb 12, 2015
cc36877
add 'id' to verbose output
teharrison Feb 13, 2015
494efbd
have auth type in datatoken
teharrison Feb 13, 2015
0c32b55
change inbox nodes to be owned by user with mgrast added to ACLs
teharrison Feb 13, 2015
e1a431f
Merge pull request #682 from teharrison/develop
teharrison Feb 13, 2015
23874a4
Merge branch 'develop' of github.com:MG-RAST/MG-RAST into develop
Feb 17, 2015
13e8622
updated to unified authHeader
Feb 17, 2015
36da604
Merge pull request #683 from paczian/develop
Feb 17, 2015
bfa9df9
added user impersonation
Feb 17, 2015
c66da37
fixed bug in impersonation
Feb 17, 2015
98a0f79
fixed typo
Feb 17, 2015
08546a1
fixed another typo
Feb 17, 2015
a834028
added token creation / refresh to impersonate
Feb 17, 2015
f928468
fixed bugs in impersonate refresh webkey
Feb 17, 2015
293952f
Merge pull request #684 from paczian/develop
Feb 17, 2015
ebef96b
added expiration to token
Feb 17, 2015
277ba4f
Merge pull request #685 from paczian/develop
Feb 17, 2015
48c8581
add awe function
teharrison Feb 17, 2015
f6174ce
add inbox awe job info to nodes, calculate file_info and refresh job …
teharrison Feb 17, 2015
2d33195
resolve conflict
teharrison Feb 17, 2015
db33bbc
correct variable name
teharrison Feb 17, 2015
972062b
added pipeline job info to input file node attributes
teharrison Feb 17, 2015
7d348fe
set type for excel files
teharrison Feb 18, 2015
1c3b454
set type for excel files
teharrison Feb 18, 2015
b15674e
add delete function
teharrison Feb 18, 2015
75a6f74
status message updates
teharrison Feb 18, 2015
e1b9f05
accept delete method
teharrison Feb 18, 2015
e9de7bf
fix filename
teharrison Feb 18, 2015
d1b731b
fix status message
teharrison Feb 18, 2015
8d9f3e4
check if stats has been ran previously
teharrison Feb 19, 2015
74cf660
update shock node after validation
teharrison Feb 19, 2015
b69b946
update output filenames
teharrison Feb 19, 2015
784c6fe
add rename and single file view, update workflows
teharrison Feb 19, 2015
7b7e1fb
typo fixes
teharrison Feb 19, 2015
2d4a6d2
fix sff2fastq output filename
teharrison Feb 20, 2015
f54477b
add seq stats calc to output of sff2fastq workflow
teharrison Feb 20, 2015
69a5548
add seq stats calc to output of pair_join workflow
teharrison Feb 20, 2015
632da5a
added view_inbox_actions method
Feb 23, 2015
4e41d7f
added state option to inbox action query
Feb 23, 2015
a3c73ac
Merge pull request #687 from paczian/develop
teharrison Feb 23, 2015
a5de3ec
attribute update
teharrison Feb 23, 2015
124a50c
add seq stats calc to output of workflows
teharrison Feb 23, 2015
d2c7afc
add seq stats calc to output of workflows
teharrison Feb 23, 2015
6671d4f
update job creation defaults
teharrison Feb 23, 2015
868995d
resolve conflicts, add pending to request list
teharrison Feb 23, 2015
3de4d16
Merge pull request #686 from teharrison/develop
teharrison Feb 23, 2015
059e606
typo
teharrison Feb 23, 2015
79e8099
Merge pull request #688 from teharrison/develop
teharrison Feb 23, 2015
d9cb289
added cite MG-RAST API button
Mar 3, 2015
e07a966
Merge pull request #689 from paczian/develop
Mar 3, 2015
393ed56
added heartbeat check for SHOCK and AWE before submission
Mar 3, 2015
739ee0a
Merge pull request #690 from paczian/develop
Mar 3, 2015
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions src/MGRAST/cgi/api.cgi
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ if (lc($request_method) eq 'options') {
-status => 200,
-type => 'text/plain',
-charset => 'UTF-8',
-Access_Control_Allow_Methods => 'POST, GET, OPTIONS, PUT',
-Access_Control_Allow_Methods => 'POST, GET, OPTIONS, PUT, DELETE',
-Access_Control_Allow_Headers => 'AUTH, AUTHORIZATION'
);
print "";
Expand Down Expand Up @@ -148,12 +148,12 @@ else {

# check for authentication
my $user;
if ($cgi->http('HTTP_AUTH') || $cgi->param('auth') || $cgi->http('HTTP_AUTHORIZATION') || $cgi->param('authorization')) {
if ($cgi->http('HTTP_AUTH') || $cgi->param('auth') || $cgi->http('HTTP_Authorization') || $cgi->param('authorization')) {
eval {
require Auth;
Auth->import();
my $message;
($user, $message) = Auth::authenticate($cgi->http('HTTP_AUTH') || $cgi->param('auth') || $cgi->http('HTTP_AUTHORIZATION') || $cgi->param('authorization'));
($user, $message) = Auth::authenticate($cgi->http('HTTP_AUTH') || $cgi->param('auth') || $cgi->http('HTTP_Authorization') || $cgi->param('authorization'));
unless($user) {
unless ($message eq "valid kbase user") {
print $cgi->header( -type => 'application/json',
Expand Down
53 changes: 33 additions & 20 deletions src/MGRAST/html/js/Upload.js
Original file line number Diff line number Diff line change
Expand Up @@ -711,26 +711,39 @@ function check_submitable () {
}

function submit_job () {
document.getElementById("submit_job_button").disabled = true;
var seq_files = selected_sequence_files.join('|');
$.ajax({ async: false, type: "POST", url: "metagenomics.cgi", data: { page: "Upload", action: "check_for_duplicates", seqfiles: seq_files }, success: function (result) {
if (result == "unique") {
document.forms.submission_form.submit();
} else if (result.match(/ERROR/) ) {
alert(result);
return false;
} else {
if ( confirm(result) ) {
// If the user allows a duplicate to be loaded into the system, we call a perl subroutine that
// will e-mail mg-rast@mcs.anl.gov if the duplicate is over 1GB in size (or whatever limit is set in Conf.pm)
$.ajax({ async: false, type: "POST", url: "metagenomics.cgi", data: { page: "Upload", action: "send_email_for_duplicate_submission", seqfiles: seq_files } });
document.forms.submission_form.submit();
} else {
document.getElementById("submit_job_button").disabled = false;
return false;
}
}
}});
// test if AWE and SHOCK are up
jQuery.getJSON( "http://api.metagenomics.anl.gov/heartbeat/AWE", function(data) {
if (data.status != 1) {
alert("Our submission pipeline is currently out of service. Please try again later.");
return;
}
jQuery.getJSON( "http://api.metagenomics.anl.gov/heartbeat/SHOCK", function(data) {
if (data.status != 1) {
alert("Our submission pipeline is currently out of service. Please try again later.");
return;
}
document.getElementById("submit_job_button").disabled = true;
var seq_files = selected_sequence_files.join('|');
$.ajax({ async: false, type: "POST", url: "metagenomics.cgi", data: { page: "Upload", action: "check_for_duplicates", seqfiles: seq_files }, success: function (result) {
if (result == "unique") {
document.forms.submission_form.submit();
} else if (result.match(/ERROR/) ) {
alert(result);
return false;
} else {
if ( confirm(result) ) {
// If the user allows a duplicate to be loaded into the system, we call a perl subroutine that
// will e-mail mg-rast@mcs.anl.gov if the duplicate is over 1GB in size (or whatever limit is set in Conf.pm)
$.ajax({ async: false, type: "POST", url: "metagenomics.cgi", data: { page: "Upload", action: "send_email_for_duplicate_submission", seqfiles: seq_files } });
document.forms.submission_form.submit();
} else {
document.getElementById("submit_job_button").disabled = false;
return false;
}
}
}});
});
});
}

function toggle (id) {
Expand Down
64 changes: 53 additions & 11 deletions src/MGRAST/lib/Auth.pm
Original file line number Diff line number Diff line change
Expand Up @@ -22,31 +22,68 @@ sub authenticate {
$key =~ s/^mggo4711//;

use MIME::Base64;
use LWP::UserAgent;
use Conf;
my ($u,$p) = split(/\:/, decode_base64($key));
my $us = $master->User->init( { login => $u } );
if (ref $us and crypt($p, $us->password) eq $us->password) {
my $pref = $master->Preferences->get_objects( { name => 'WebServiceKeyTdate', user => $us } );
if (scalar(@$pref)) {
if (scalar(@$pref)) {
if ($pref->[0]->value < time) {
$pref->[0]->value(time + 1209600);
}
my $skeytimeout = $pref->[0]->value();
$pref = $master->Preferences->get_objects( { name => 'WebServicesKey', user => $us } );
my $cgi = new CGI;
my $verbose = "";
if ($cgi->param('verbosity') && $cgi->param('verbosity') eq 'verbose') {
$verbose = ', "login": "'.$us->{login}.'", "firstname": "'.$us->{firstname}.'", "lastname": "'.$us->{lastname}.'", "email": "'.$us->{email}.'"';
}
my $cgi = new CGI;
my $verbose = "";
if ($cgi->param('verbosity') && $cgi->param('verbosity') eq 'verbose') {
$verbose = ', "login":"'.$us->{login}.'", "firstname":"'.$us->{firstname}.'", "lastname":"'.$us->{lastname}.'", "email":"'.$us->{email}.'", "id":"mgu'.$us->{_id}.'"';
# SHOCK preferences
my $prefs = $master->Preferences->get_objects({ user => $us, name => "shock_pref_node" });
if (scalar(@$prefs)) {
my $nodeid = $prefs->[0]->{value};
my $response = undef;
my $json = new JSON;
$json = $json->utf8();
$json->max_size(0);
$json->allow_nonref;
my $agent = LWP::UserAgent->new;
eval {
my @args = ('Authorization', "mgrast ".$pref->[0]->{value});
my $url = $Conf::shock_url.'/node/'.$nodeid;
my $get = $agent->get($url, @args);
$response = $json->decode( $get->content );
};
if ($@ || (! ref($response))) {
print $cgi->header(-type => 'application/json',
-status => 500,
-charset => 'UTF-8',
-Access_Control_Allow_Origin => '*' );
print $json->encode({"ERROR" => "Unable to GET node $nodeid from Shock: ".$response->{error}[0]}, $response->{status} );
exit;
} elsif (exists($response->{error}) && $response->{error}) {
print $cgi->header(-type => 'application/json',
-status => 500,
-charset => 'UTF-8',
-Access_Control_Allow_Origin => '*' );
print $json->encode({"ERROR" => "Unable to GET node $nodeid from Shock: ".$response->{error}[0]}, $response->{status} );
exit;
} else {
$verbose.=', "preferences": '.$json->encode($response->{data}->{attributes}->{pref});
}
}
}
print $cgi->header(-type => 'application/json',
-status => 200,
-charset => 'UTF-8',
-charset => 'UTF-8',
-Access_Control_Allow_Origin => '*' );
print '{ "token": "'.$pref->[0]->value.'"'.$verbose.' }';
print '{ "token": "'.$pref->[0]->value.'", "expiration": "'.$skeytimeout.'"'.$verbose.' }';
exit;
} else {
return (undef, "api access not enabled for this user");
}
return (undef, "api access not enabled for this user");
}
} else {
return (undef, "invalid MG-RAST credentials");
return (undef, "invalid MG-RAST credentials");
}
}

Expand Down Expand Up @@ -176,6 +213,11 @@ sub authenticate {
}
}
}

# check for MG-RAST default auth header
if ($auth_value =~ /^mgrast /) {
$auth_value =~ s/^mgrast //;
}

# check for the preference setting for the defined authentication source and value
my $preference = $master->Preferences->get_objects( { name => $auth_source, value => $auth_value } );
Expand Down
22 changes: 9 additions & 13 deletions src/MGRAST/lib/GoogleAnalytics.pm
Original file line number Diff line number Diff line change
Expand Up @@ -41,24 +41,20 @@ sub track_page_view {

my $visitor_id = $uid || &get_random_number();

my $utm_gif_location = "http://www.google-analytics.com/__utm.gif";
my $url = "http://www.google-analytics.com/collect";

my $remote_address = "";
if (exists($ENV{'REMOTE_ADDR'})) {
$remote_address = $ENV{'REMOTE_ADDR'};
}

# Construct the gif hit url.
my $utm_url = $utm_gif_location . '?' .
'utmwv=' . VERSION .
'&utmn=' . get_random_number() .
'&utmhn=' . uri_escape($domain_name) .
'&utmr=' . uri_escape($document_referer) .
'&utmp=' . uri_escape($document_path) .
'&utmac=' . $account .
'&utmcc=__utma%3D999.999.999.999.999.1%3B' .
'&utmvid=' . $visitor_id .
'&utmip=' . anonymize_ip($remote_address);
# Construct the payload
my $content = 'v=' . VERSION .
'&tid=' . $account .
'&cid=' . anonymize_ip($remote_address) .
'&t=pageview' .
'&vid=' . $visitor_id .
'&dl=' . uri_escape($document_path);

my $ua = LWP::UserAgent->new;

Expand All @@ -69,7 +65,7 @@ sub track_page_view {
$ua->agent($ENV{'HTTP_USER_AGENT'});
}

my $ga_output = $ua->get($utm_url);
my $ga_output = $ua->post($url, Content => $content);

if (defined($debug) && !$ga_output->is_success) {
print STDERR $ga_output->status_line;
Expand Down
10 changes: 5 additions & 5 deletions src/MGRAST/lib/WebPage/Analysis.pm
Original file line number Diff line number Diff line change
Expand Up @@ -600,8 +600,8 @@ sub single_data {
if ($mg_grp_sel eq 'groups') {
my $joined_data = {};
foreach my $row (@$result) {
my $id_string = join("|", @$row[1..8]);
$row->[0] = $collections->{$row->[0]};
my $id_string = join("|", @$row[0..8]);
if (exists($joined_data->{$id_string})) {
$row->[10] = sprintf("%.2f", (($joined_data->{$id_string}->[9] * $joined_data->{$id_string}->[10]) + ($row->[9] * $row->[10])) / ($joined_data->{$id_string}->[9] + $row->[9]));
$row->[11] = sprintf("%.2f", (($joined_data->{$id_string}->[9] * $joined_data->{$id_string}->[11]) + ($row->[9] * $row->[11])) / ($joined_data->{$id_string}->[9] + $row->[9]));
Expand Down Expand Up @@ -715,8 +715,8 @@ sub phylogenetic_data {
if ($mg_grp_sel eq 'groups') {
my $joined_data = {};
foreach my $row (@$result) {
my $id_string = join("|", @$row[1..9]);
$row->[0] = $collections->{$row->[0]};
my $id_string = join("|", @$row[0..9]);
if (exists($joined_data->{$id_string})) {
$row->[12] = sprintf("%.2f", (($joined_data->{$id_string}->[10] * $joined_data->{$id_string}->[12]) + ($row->[10] * $row->[12])) / ($joined_data->{$id_string}->[10] + $row->[10]));
$row->[13] = sprintf("%.2f", (($joined_data->{$id_string}->[10] * $joined_data->{$id_string}->[13]) + ($row->[10] * $row->[13])) / ($joined_data->{$id_string}->[10] + $row->[10]));
Expand Down Expand Up @@ -861,8 +861,8 @@ sub metabolic_data {
if ($mg_grp_sel eq 'groups') {
my $joined_data = {};
foreach my $row (@$all) {
my $id_string = join("|", @$row[1..4]);
$row->[0] = $collections->{$row->[0]};
my $id_string = join("|", @$row[0..4]);
if (exists($joined_data->{$id_string})) {
$row->[8] = sprintf("%.2f", (($joined_data->{$id_string}->[6] * $joined_data->{$id_string}->[8]) + ($row->[6] * $row->[8])) / ($joined_data->{$id_string}->[6] + $row->[6]));
$row->[9] = sprintf("%.2f", (($joined_data->{$id_string}->[6] * $joined_data->{$id_string}->[9]) + ($row->[6] * $row->[9])) / ($joined_data->{$id_string}->[6] + $row->[6]));
Expand Down Expand Up @@ -966,8 +966,8 @@ sub annotation_data {
if ($mg_grp_sel eq 'groups') {
my $joined_data = {};
foreach my $row (@$result) {
my $id_string = join("|", @$row[1..2]);
$row->[0] = $collections->{$row->[0]};
my $id_string = join("|", @$row[0..2]);
if (exists($joined_data->{$id_string})) {
$row->[5] = sprintf("%.2f", (($joined_data->{$id_string}->[3] * $joined_data->{$id_string}->[5]) + ($row->[3] * $row->[5])) / ($joined_data->{$id_string}->[3] + $row->[3]));
$row->[6] = sprintf("%.2f", (($joined_data->{$id_string}->[3] * $joined_data->{$id_string}->[6]) + ($row->[3] * $row->[6])) / ($joined_data->{$id_string}->[3] + $row->[3]));
Expand Down Expand Up @@ -1053,8 +1053,8 @@ sub lca_data {
if ($mg_grp_sel eq 'groups') {
my $joined_data = {};
foreach my $row (@$result) {
my $id_string = join("|", @$row[1..8]);
$row->[0] = $collections->{$row->[0]};
my $id_string = join("|", @$row[0..8]);
if (exists($joined_data->{$id_string})) {
$row->[10] = sprintf("%.2f", (($joined_data->{$id_string}->[9] * $joined_data->{$id_string}->[10]) + ($row->[9] * $row->[10])) / ($joined_data->{$id_string}->[9] + $row->[9]));
$row->[11] = sprintf("%.2f", (($joined_data->{$id_string}->[9] * $joined_data->{$id_string}->[11]) + ($row->[9] * $row->[11])) / ($joined_data->{$id_string}->[9] + $row->[9]));
Expand Down
2 changes: 1 addition & 1 deletion src/MGRAST/lib/WebPage/Home.pm
Original file line number Diff line number Diff line change
Expand Up @@ -174,7 +174,7 @@ END

$content .= "<p style='color:#8FBC3F;text-align:left;font-size:9px;margin-top:3px;'>This work was supported in part by the Office of Advanced Scientific Computing Research, Office of Science, U.S. Department of Energy, under Contract DE-AC02-06CH11357.</p>";

$content .= "<a href=\"http://www.biomedcentral.com/1471-2105/9/386\" target=\"_blank\"><p style='border-radius: 5px; -webkit-border-radius: 5px; -moz-border-radius: 5px; color: white; padding: 5px 0px 3px 10px; color: white; font-size:14; background-color: #5281B0; width: 100px; float: left; margin-right: 10px; top: -12px; position: relative;'>cite MG-RAST</p></a>";
$content .= "<a href=\"http://www.biomedcentral.com/1471-2105/9/386\" target=\"_blank\"><p style='border-radius: 5px; -webkit-border-radius: 5px; -moz-border-radius: 5px; color: white; padding: 5px 0px 3px 10px; color: white; font-size:14; background-color: #5281B0; width: 100px; float: left; margin-right: 10px; top: -12px; position: relative;'>cite MG-RAST</p></a><a href=\"http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1004008\" target=\"_blank\" style=\"float: right;\"><p style='border-radius: 5px; -webkit-border-radius: 5px; -moz-border-radius: 5px; color: white; padding: 5px 0px 3px 10px; color: white; font-size:14; background-color: #5281B0; width: 120px; float: left; margin-right: 10px; top: -12px; position: relative;'>cite MG-RAST API</p></a>";

#$content .= "<p style='color:#8FBC3F;text-align:left;font-size:9px;margin-top:3px;'><em>The Metagenomics RAST server - A public resource for the automatic phylogenetic and functional analysis of metagenomes</em> F. Meyer, D. Paarmann, M. D'Souza, R. Olson , E. M. Glass, M. Kubal, T. Paczian , A. Rodriguez , R. Stevens, A. Wilke, J. Wilkening, R. A. Edwards<br/><em>BMC Bioinformatics 2008, 9:386
#, [<a href=\"http://www.biomedcentral.com/1471-2105/9/386\" target=\"_blank\">article</a>]</em></p>";
Expand Down
3 changes: 3 additions & 0 deletions src/MGRAST/lib/resources/annotation.pm
Original file line number Diff line number Diff line change
Expand Up @@ -345,6 +345,9 @@ sub print_batch {
$fields = [$solr_key, 'md5_id'];
}
} elsif ($type eq 'feature') {
$solr_key = 'accession';
$fields = [$solr_key, 'md5_id'];
} else {
$fields = ['md5_id'];
}

Expand Down
Loading