How to login to APEX190200 application using R web-scraping (e.g. POST request)

64 views Asked by At

I'm not an expert in authentication stuff.

I have been tasked with fixing an R shiny app (not mine) that performs some web-scraping on an Oracle APEX application. The login procedure worked in the past when APEX050000 was in use, but started failing with APEX was updated to version 190200.

The Oracle APEX change log shows a potentially interesting change in version 18.1:

Social Authentication

Developers can now easily create APEX applications which can use Oracle Identity Cloud Service, Google, Facebook, generic OpenID Connect and generic OAuth2 as the authentication method.

But I can't judge if that could have made the difference.

In the R shiny app, the form is submitted with a POST request (is it BASIC auth?). The code performs steps I cant explain, so I hope it speaks for itself. I simplified where I could:

library(rvest)
library(httr)

dv.login <- function(username, password){
  
  url = "http://ufo.some.url.nl/apex/f?p=200:LOGIN_DESKTOP::::::"
  dv_session <<- dv_session %>% rvest:::request_GET(url)
  
  form <- dv_session %>% html_nodes("form")
  action <- form %>% html_attr("action")  # wwv_flow.accept
  inputs <- form %>% html_nodes("input")
  
  for (inp in inputs) { 
    name <- inp %>% html_attr("name", default = "-")
    if (name == "-") {next}
    value <- input %>% html_nodes("option[selected=selected]") %>% html_attr("value")
    
    keys <- c(keys, name)
    values <- c(values, value)
  }
  post_params <- as.list(values)
  names(post_params) <- keys
  
  post_params["p_md5_checksum"]  <- dv_session %>% html_nodes("input[name=p_md5_checksum]") %>% html_attr("value")
  post_params["p_page_checksum"] <- dv_session %>% html_nodes("input[name=p_page_checksum]") %>% html_attr("value")
  post_params["p_request"] <- "LOGIN"
  post_params["p_t01"] <- username
  post_params["p_t02"] <- password
  
  url_post = "http://ufo.some.url.nl/apex/wwv_flow.accept"
  dv.POST_request(url_post, post_params)
  dv_session <<- dv_session %>%
    rvest:::request_POST(url_post,
                         body = post_params,
                         encode = "form")
  
  if (username %in% dv_session$response$cookies$value) {
    instance <<- post_params$p_instance
    
    username <<- username
    password <<- password
    login_correct <<- TRUE
    return('succes')
  }
  login_error <<- "Wrong username or password"
  return('fail')
}

Issue

The "p_md5_checksum" and "p_page_checksum" values cannot be found anymore! I tried to find alternative html nodes that contain these checksums, but it seems it is not there... In other words, the node matching the xpath input[name=p_md5_checksum] does not exist anymore.

I tried it with the pPageItemsProtected and pPageFormRegionChecksums values of respectively [] and Gnxyk3e2EUyHfpQDZWCMtQ, but received a 404 - page not found (If I remember correctly).

Beside this primary issue, I doubt if the password and username post_param fields are correct, they were p_t01 and p_t02, but that's not something I can find in the HTML... should it be there?

HTML

I could obtain the HTML by running:

httr::content(dv.session$response, as="text")

(I don't have a browser's "webdeveloper tools" because I can only access it from a restricted Citrix environment where that's disabled)

Result:

<html lang="en" xmlns:htmldb="http://htmldb.oracle.com">

<head>
  <meta http-equiv="x-ua-compatible" content="IE=edge" />
  <title>Login</title>
  <link rel="stylesheet" href="/i/themes/theme_20/theme_4_0.css" type="text/css" />
  <!--[if IE]><link rel="stylesheet" href="/i/themes/theme_20/ie.css" type="text/css" /><![endif]-->
  <link rel="stylesheet" href="/i/app_ui/css/Core.min.css?v=19.2.0.00.18" type="text/css" />
  <link rel="stylesheet" href="/i/app_ui/css/Theme-Standard.min.css?v=19.2.0.00.18" type="text/css" />
  <link rel="stylesheet" href="/i/libraries/jquery-ui/1.12.1/jquery-ui-apex.min.css?v=19.2.0.00.18" type="text/css" />

  <link rel="stylesheet" href="/i/legacy_ui/css/5.0.min.css?v=19.2.0.00.18" type="text/css" />



  <script>
    var apex_img_dir = "/i/",
      htmldb_Img_Dir = apex_img_dir;
  </script>
  <script src="/i/libraries/apex/minified/desktop_all.min.js?v=19.2.0.00.18"></script>
  <script src="wwv_flow.js_messages?p_app_id=200&p_lang=en&p_version=19.2.0.00.18-10662680120535"></script>
  <script src="/i/libraries/apex/minified/legacy_pre18.min.js?v=19.2.0.00.18"></script>
  <script src="/i/libraries/apex/minified/legacy_18.min.js?v=19.2.0.00.18"></script>
  <script src="/i/libraries/jquery-migrate/3.0.1/jquery-migrate-3.0.1.min.js?v=19.2.0.00.18"></script>

  <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
  <meta http-equiv="Pragma" content="no-cache" />
  <meta http-equiv="Expires" content="-1" />
  <meta http-equiv="Cache-Control" content="no-cache" />
</head>

<body>
  <form action="wwv_flow.accept" method="post" name="wwv_flow" id="wwvFlowForm" data-oj-binding-provider="none" novalidate autocomplete="off">

    <input type="hidden" name="p_flow_id" value="200" id="pFlowId" />
    <input type="hidden" name="p_flow_step_id" value="101" id="pFlowStepId" />
    <input type="hidden" name="p_instance" value="10532003874288" id="pInstance" />
    <input type="hidden" name="p_page_submission_id" value="321620263451736796746631187861272947304" id="pPageSubmissionId" />
    <input type="hidden" name="p_request" value="" id="pRequest" />
    <input type="hidden" name="p_reload_on_submit" value="A" id="pReloadOnSubmit" />
    <input type="hidden" value="321620263451736796746631187861272947304" id="pSalt" />
    <div id="t20PageHeader">
      <table border="0" cellpadding="0" cellspacing="0" summary="">
        <tr>
          <td id="t20Logo" valign="top"><span class="apex-logo-text">some_tool</span><br /></td>
          <td id="t20HeaderMiddle" valign="top" width="100%"><br /></td>
          <td id="t20NavBar" valign="top"><a href="apex_authentication.logout&#x3F;p_app_id&#x3D;200&#x26;p_session_id&#x3D;10532003874288" class="t20NavBar">Logout</a> |<br /></td>
        </tr>
      </table>
    </div>
    <div id="t20BreadCrumbsLeft"></div>
    <table border="0" cellpadding="0" cellspacing="0" summary="" id="t20PageBody" height="70%" align="center" width="400">
      <td width="100%" valign="top" height="100%" id="t20ContentBody" align="center">
        <div id="t20Messages"><span id="APEX_SUCCESS_MESSAGE" data-template-id="59041209344594598_S" class="apex-page-success u-hidden"></span><span id="APEX_ERROR_MESSAGE" data-template-id="59041209344594598_E" class="apex-page-error u-hidden"></span></div>
        <div id="t20ContentMiddle">
          <table class="t20Region t20FormRegion" id="R59048808018594617" border="0" cellpadding="0" cellspacing="0" summary="">
            <thead>
              <tr>
                <th class="t20RegionHeader" id="R59048808018594617_header">Login</th>
              </tr>
            </thead>
            <tbody id="R59048808018594617_body">
              <tr>
                <td class="t20ButtonHolder"></td>
              </tr>
              <tr>
                <td class="t20RegionBody">
                  <table id="apex_layout_59048808018594617" border="0" class="formlayout" role="presentation">
                    <tr>
                      <td align="right"><label for="P101_USERNAME" id="P101_USERNAME_LABEL" tabindex="999"><a class="t20OptionalLabelwithHelp" href="javascript:popupFieldHelp('59048902184594617','10532003874288')" tabindex="999">Username</a></label></td>
                      <td align="left"><input type="text" id="P101_USERNAME" name="P101_USERNAME" class="text_field&#x20;apex-item-text" value="" size="40" maxlength="100" /></td>
                    </tr>
                    <tr>
                      <td align="right"><label for="P101_PASSWORD" id="P101_PASSWORD_LABEL" tabindex="999"><a class="t20OptionalLabelwithHelp" href="javascript:popupFieldHelp('59049017764594617','10532003874288')" tabindex="999">Password</a></label></td>
                      <td align="left"><input type="password" name="P101_PASSWORD" size="40" maxlength="100" value="" id="P101_PASSWORD" class="password&#x20;apex-item-text" onkeypress="return apex.submit({request:'P101_PASSWORD',submitIfEnter:event})" /></td>
                      <td align="left"><a href="javascript:apex.submit(%7Brequest:&#x27;LOGIN&#x27;%7D);" class="t20Button" id="P101_LOGIN">Login</a></td>
                    </tr>
                  </table>
                </td>
              </tr>
            </tbody>
          </table>
        </div>
      </td>
      <td valign="top" width="200" id="t20ContentRight"><br /></td>
      </tr>
    </table>
    <table border="0" cellpadding="0" cellspacing="0" summary="" id="t20PageFooter" width="100%">
      <tr>
        <td id="t20Left" valign="top"><span id="t20UserPrompt">nobody</span><br /></td>
        <td id="t20Center" valign="top"></td>
        <td id="t20Right" valign="top"><span id="t20Customize"></span><br /></td>
      </tr>
    </table>
    <br class="t20Break" />
    <input type="hidden" id="pPageFormRegionChecksums" value="&#x5B;&#x5D;">
    <input type="hidden" id="pPageItemsRowVersion" value="" />
    <input type="hidden" id="pPageItemsProtected" value="Gnxyk3e2EUyHfpQDZWCMtQ" /></form>







  <script type="text/javascript">
    apex.jQuery(function() {
      apex.page.init(this, function() {
        apex.jQuery.when.apply(apex.jQuery, apex.page.loadingDeferreds).done(function() {
          try {
            (function() {
              var lTimeoutField = document.getElementById("apex_login_throttle_sec"),
                lTimeout = lTimeoutField ? +lTimeoutField.innerHTML : 0;
              if (lTimeout) {
                var lTimer = window.setInterval(
                  function() {
                    if (lTimeout > 0) {
                      lTimeoutField.innerHTML = lTimeout;
                      lTimeout--;
                    } else {
                      window.clearInterval(lTimer);
                      var lDiv = document.getElementById("apex_login_throttle_div");
                      if (lDiv) {
                        lDiv.parentNode.removeChild(lDiv);
                        return true;
                      }
                    }
                  },
                  1000);
              }
            })();


            apex.item('P101_USERNAME').setFocus();
          } finally {
            apex.event.trigger(apex.gPageContext$, 'apexreadyend');
          }
        });
      });
    });
  </script>

</body>

</html>

0

There are 0 answers