You are not logged in.

Pages: 1

#1 2019-03-03 11:55:00

Lone_Wolf
Administrator
From: Netherlands, Europe
Registered: 2005-10-04
Posts: 12,930

[Solved] makepkg/curl unable to download file

https://aur.archlinux.org/packages/seventhsense wrote:

Lone_Wolf commented on 2019-03-02 13:01

Your package seventhsense [1] has been flagged out-of-date by ChinaskiJr [2]:

Downloading zip package returns 404 error on project Aon website

The file can be downloaded manually fine, but not from the PKGBUILD atm.

left-click the last link under sources (3) to get it, save the file in the folder where you're building (or the folder you have set with $SRCDEST in makepkg.conf )

The file is located at http://www.projectaon.org/staff/david/c … k.php?id=4

wget has no trouble downloading it, and also shows the real location [1]
curl chokes on all variants i can think of [2] .

firefox and wget have no trouble downloading the file, curl has.
I'm not sure whether the problem is caused by  an upstream filename with spaces, the redirects or something else.
It has been years since I last build this package, it may very well be that pacman/makepkg download agents changed in that time.

The issue with the package can be easily solved with a custom download agent using wget, but I do think there's an underlying issue here that should be reported somewhere.
Thoughts ?







[1]

$ wget http://www.projectaon.org/staff/david/click/click.php?id=4
URL transformed to HTTPS due to an HSTS policy
--2019-03-03 12:35:29--  https://www.projectaon.org/staff/david/click/click.php?id=4
Loaded CA certificate '/etc/ssl/certs/ca-certificates.crt'
Resolving www.projectaon.org (www.projectaon.org)... 69.163.153.189
Connecting to www.projectaon.org (www.projectaon.org)|69.163.153.189|:443... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: http://www.projectaon.org/staff/david/Seventh Sense Source.zip [following]
URL transformed to HTTPS due to an HSTS policy
--2019-03-03 12:35:29--  https://www.projectaon.org/staff/david/Seventh%20Sense%20Source.zip
Reusing existing connection to www.projectaon.org:443.
HTTP request sent, awaiting response... 200 OK
Length: 53691324 (51M) [application/zip]
Saving to: ‘click.php?id=4.2’

click.php?id=4.2                      100%[=======================================================================>]  51.20M  2.57MB/s    in 23s     

2019-03-03 12:35:53 (2.21 MB/s) - ‘click.php?id=4.2’ saved [53691324/53691324]

$

[2]

$ curl -v http://www.projectaon.org/staff/david/click/click.php?id=4
*   Trying 69.163.153.189...
* TCP_NODELAY set
* Connected to www.projectaon.org (69.163.153.189) port 80 (#0)
> GET /staff/david/click/click.php?id=4 HTTP/1.1
> Host: www.projectaon.org
> User-Agent: curl/7.64.0
> Accept: */*
> 
< HTTP/1.1 301 Moved Permanently
< Date: Sun, 03 Mar 2019 11:38:18 GMT
< Server: Apache
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< X-Xss-Protection: 1; mode=block
< X-Frame-Options: SAMEORIGIN
< X-Content-Type-Options: nosniff
< Location: https://www.projectaon.org/staff/david/click/click.php?id=4
< Vary: Accept-Encoding
< Content-Length: 267
< Content-Type: text/html; charset=iso-8859-1
< 
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a href="https://www.projectaon.org/staff/david/click/click.php?id=4">here</a>.</p>
</body></html>
* Connection #0 to host www.projectaon.org left intact




$ curl -v http://www.projectaon.org/staff/david/Seventh Sense Source.zip
*   Trying 69.163.153.189...
* TCP_NODELAY set
* Connected to www.projectaon.org (69.163.153.189) port 80 (#0)
> GET /staff/david/Seventh HTTP/1.1
> Host: www.projectaon.org
> User-Agent: curl/7.64.0
> Accept: */*
> 
< HTTP/1.1 301 Moved Permanently
< Date: Sun, 03 Mar 2019 11:39:06 GMT
< Server: Apache
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< X-Xss-Protection: 1; mode=block
< X-Frame-Options: SAMEORIGIN
< X-Content-Type-Options: nosniff
< Location: https://www.projectaon.org/staff/david/Seventh
< Vary: Accept-Encoding
< Content-Length: 254
< Content-Type: text/html; charset=iso-8859-1
< 
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a href="https://www.projectaon.org/staff/david/Seventh">here</a>.</p>
</body></html>
* Connection #0 to host www.projectaon.org left intact
* Could not resolve host: Sense
* Closing connection 1
curl: (6) Could not resolve host: Sense
* Could not resolve host: Source.zip
* Closing connection 2
curl: (6) Could not resolve host: Source.zip




$ curl -v https://www.projectaon.org/staff/david/Seventh%20Sense%20Source.zip
*   Trying 69.163.153.189...
* TCP_NODELAY set
* Connected to www.projectaon.org (69.163.153.189) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=projectaon.org
*  start date: Feb  7 12:02:13 2019 GMT
*  expire date: May  8 12:02:13 2019 GMT
*  subjectAltName: host "www.projectaon.org" matched cert's "www.projectaon.org"
*  issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
*  SSL certificate verify ok.
> GET /staff/david/Seventh%20Sense%20Source.zip HTTP/1.1
> Host: www.projectaon.org
> User-Agent: curl/7.64.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Date: Sun, 03 Mar 2019 11:40:06 GMT
< Server: Apache
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< X-Xss-Protection: 1; mode=block
< X-Frame-Options: SAMEORIGIN
< X-Content-Type-Options: nosniff
< Last-Modified: Mon, 26 Oct 2015 14:25:23 GMT
< ETag: "33343bc-52302bb4f5ac0"
< Accept-Ranges: bytes
< Content-Length: 53691324
< Content-Type: application/zip
< 
Warning: Binary output can mess up your terminal. Use "--output -" to tell 
Warning: curl to output it to your terminal anyway, or consider "--output 
Warning: <FILE>" to save to a file.
* Failed writing body (0 != 16384)
* Closing connection 0



$ curl -v "https://www.projectaon.org/staff/david/Seventh Sense Source.zip"
*   Trying 69.163.153.189...
* TCP_NODELAY set
* Connected to www.projectaon.org (69.163.153.189) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=projectaon.org
*  start date: Feb  7 12:02:13 2019 GMT
*  expire date: May  8 12:02:13 2019 GMT
*  subjectAltName: host "www.projectaon.org" matched cert's "www.projectaon.org"
*  issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
*  SSL certificate verify ok.
> GET /staff/david/Seventh Sense Source.zip HTTP/1.1
> Host: www.projectaon.org
> User-Agent: curl/7.64.0
> Accept: */*
> 
< HTTP/1.1 404 Not Found
< Date: Sun, 03 Mar 2019 11:40:54 GMT
< Server: Apache
< Expires: Tue, 01 Jan 2002 00:00:00 GMT
< Cache-Control: no-cache
< Set-Cookie: imstime=1551613254; expires=Tue, 02-Apr-2019 11:40:54 GMT; Max-Age=2592000; path=/
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< X-Xss-Protection: 1; mode=block
< X-Frame-Options: SAMEORIGIN
< X-Content-Type-Options: nosniff
< Last-Modified: Sun, 03 Mar 2019 11:40:54 GMT
< Connection: close
< Content-Type: text/html; charset=UTF-8
< 
<!DOCTYPE html>
<html lang="en">
  <head>
    <title>Project Aon: Errors / 404 (browse)</title>
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/css/bootstrap.min.css" type="text/css" />
    <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.3/jquery.min.js"></script>
    <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/js/bootstrap.min.js"></script>
    <!--HTMLHeader--><meta http-equiv='Content-Type' content='text/html; charset=utf-8' />        <link rel="alternate" type="application/rss+xml" href="https://www.projectaon.org/en/blog/category/whats-new/feed/" title="What's New - Kai Grand Sentinel" />
<script type="text/javascript">
function toggle(obj, hide, show) {
    var elstyle = document.getElementById(obj).style;
    var text    = document.getElementById(obj + "tog");
    if(!hide) { var hide = "hide"; }
    if(!show) { var show = "show"; }
    if (elstyle.display == 'none') {
        elstyle.display = 'block';
        text.innerHTML  = hide;
    } else {
        elstyle.display = 'none';
        text.innerHTML  = show;
    }
}
</script>    <link rel="stylesheet" href="https://www.projectaon.org/en/pub/skins/ishir/styles/main.css" type="text/css" />
    <link rel="stylesheet" href="https://www.projectaon.org/en/pub/skins/ishir/styles/wiki.css" type="text/css" />
    <link rel="shortcut icon" href="/favicon.ico" />

  </head>
  <body>
    <div class="container">
      <header id="main-header">
        <h1><a href="https://www.projectaon.org/en"><span>Project Aon</span></a></h1>
      </header>

      <nav id="main-navbar" class="navbar navbar-ishir">
          <button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#main-navbar-collapse">
            <span class="icon-bar"></span>
            <span class="icon-bar"></span>
            <span class="icon-bar"></span>
          </button>
        <div class="collapse navbar-collapse" id="main-navbar-collapse">
          <ul class='nav navbar-nav'><li> <a href='https://www.projectaon.org/en/Main/Home'>Home</a>
</li><li><a href='https://www.projectaon.org/en/Main/Books'>Books</a>
</li><li><a href='https://www.projectaon.org/en/Main/WhatsNew'>News</a>
</li><li><a href='https://www.projectaon.org/en/Main/FAQ'>FAQ</a>
</li><li><a href='http://projectaon.proboards.com/' rel='nofollow'>Forum</a>
</li><li><a href='https://www.projectaon.org/en/Main/Links'>Links</a>
</li><li><a href='https://www.projectaon.org/en/Main/HelpUs'>Help Us</a>
</li></ul>
 
        </div>
      </nav>

      <article>
         <!--PageText-->
<div id='wikitext'>
<h2>404 - Not Found</h2>
<p>The requested URL was not found on this server. If the link that you followed was on the Project Aon site, please <a href='https://www.projectaon.org/en/Main/Contact'>contact us</a> so that we can correct the error. We apologize for the annoyance.
</p>
<p>Your life and quest continue <a href='http://www.projectaon.org/' rel='nofollow'>here</a>.
</p>
</div>

      </article>

      <footer>
       <div id="footer-top" class="row">
         <div id="page-info" class="col-sm-5">
           <form id="search-form" class="form-inline" role="form" action="https://www.projectaon.org/en">
             <input type="hidden" name="n" value="Errors.404" />
             <input type="hidden" name="group" value="Errors" />
             <input type="hidden" name="action" value="search" />
             <input type="text" name="q" class="searchbox form-control" />
             <input type="submit" class="btn" value="Search" />
           </form>
           <p>Updated: March 20, 2006, at 11:03 AM</p>
         </div>
         <div class="col-sm-3">
           <p><a href="https://www.projectaon.org/en//Main/Contact">Contact Us</a></p>
         </div>
         <div id="language-selection" class="col-sm-4">
           <p class='list-group'>  <a class='self-language-link list-group-item' href='https://www.projectaon.org/en/' rel='nofollow'>English</a>
 <a class='list-group-item' href='https://www.projectaon.org/es/' rel='nofollow'>Español</a>
 <a class='list-group-item' href='https://www.projectaon.org/it/' rel='nofollow'>Italiano</a>
</p>
 
         </div>
       </div>
       <nav id="page-actions" class="navbar navbar-ishir">
        <button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#page-actions-navbar-collapse">
          <span class="icon-bar"></span>
          <span class="icon-bar"></span>
          <span class="icon-bar"></span>
        </button>
        <div class="collapse navbar-collapse" id="page-actions-navbar-collapse">
         <ul class='nav navbar-nav'><li>       <a accesskey=''  rel='nofollow'  class='selflink' href='https://www.projectaon.org/en/Errors/404'>View</a>
</li><li>      <a accesskey='e'  rel='nofollow'  href='https://www.projectaon.org/en/Errors/404?action=edit'>Edit</a>
</li><li>   <a accesskey='h'  rel='nofollow'  href='https://www.projectaon.org/en/Errors/404?action=diff'>History</a>
</li><li>     <a accesskey=''  rel='nofollow'  href='https://www.projectaon.org/en/Errors/404?action=print'>Print</a>
</li></ul>

        </div>
       </nav>
      </footer>
    </div>
  </body> 
</html>
* TLSv1.2 (IN), TLS alert, close notify (256):
* Closing connection 0
$

Last edited by Lone_Wolf (2019-03-05 12:51:53)

Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.

clean chroot building not flexible enough ?
Try clean chroot manager by graysky

Offline

#2 2019-03-03 12:30:24

loqs
Member
Registered: 2014-03-06
Posts: 18,045

Re: [Solved] makepkg/curl unable to download file

curl -v https://www.projectaon.org/staff/david/Seventh%20Sense%20Source.zip

Offline

#3 2019-03-03 12:39:28

schard
Forum Moderator
From: Hannover
Registered: 2016-05-06
Posts: 2,111
Website

Re: [Solved] makepkg/curl unable to download file

I.e. curl does not automatically handle 301 redirections.
Use curl -L.

Inofficial first vice president of the Rust Evangelism Strike Force

Offline

#4 2019-03-03 12:44:23

loqs
Member
Registered: 2014-03-06
Posts: 18,045

Re: [Solved] makepkg/curl unable to download file

schard wrote:

I.e. curl does not automatically handle 301 redirections.
Use curl -L.

curl -L -v https://www.projectaon.org/staff/david/click/click.php?id=4
*   Trying 69.163.153.189...
* TCP_NODELAY set
* Connected to www.projectaon.org (69.163.153.189) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=projectaon.org
*  start date: Feb  7 12:02:13 2019 GMT
*  expire date: May  8 12:02:13 2019 GMT
*  subjectAltName: host "www.projectaon.org" matched cert's "www.projectaon.org"
*  issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
*  SSL certificate verify ok.
> GET /staff/david/click/click.php?id=4 HTTP/1.1
> Host: www.projectaon.org
> User-Agent: curl/7.64.0
> Accept: */*
> 
< HTTP/1.1 302 Moved Temporarily
< Date: Sun, 03 Mar 2019 12:43:28 GMT
< Server: Apache
< P3P: CP="NOI NID"
< Set-Cookie: ccount_unique=4; expires=Sun, 03-Mar-2019 18:43:28 GMT; Max-Age=21600
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< X-Xss-Protection: 1; mode=block
< X-Frame-Options: SAMEORIGIN
< X-Content-Type-Options: nosniff
< Location: http://www.projectaon.org/staff/david/Seventh Sense Source.zip
< Content-Length: 0
< Content-Type: text/html; charset=UTF-8
< 
* Connection #0 to host www.projectaon.org left intact
* Issue another request to this URL: 'http://www.projectaon.org/staff/david/Seventh Sense Source.zip'
*   Trying 69.163.153.189...
* TCP_NODELAY set
* Connected to www.projectaon.org (69.163.153.189) port 80 (#1)
> GET /staff/david/Seventh Sense Source.zip HTTP/1.1
> Host: www.projectaon.org
> User-Agent: curl/7.64.0
> Accept: */*
> 
< HTTP/1.1 301 Moved Permanently
< Date: Sun, 03 Mar 2019 12:43:29 GMT
< Server: Apache
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< X-Xss-Protection: 1; mode=block
< X-Frame-Options: SAMEORIGIN
< X-Content-Type-Options: nosniff
< Location: https://www.projectaon.org/staff/david/Seventh
< Vary: Accept-Encoding
< Content-Length: 254
< Connection: close
< Content-Type: text/html; charset=iso-8859-1
< 
* Closing connection 1
* Issue another request to this URL: 'https://www.projectaon.org/staff/david/Seventh'
* Found bundle for host www.projectaon.org: 0x556ebd62e0d0 [can pipeline]
* Could pipeline, but not asked to!
* Re-using existing connection! (#0) with host www.projectaon.org
* Connected to www.projectaon.org (69.163.153.189) port 443 (#0)
> GET /staff/david/Seventh HTTP/1.1
> Host: www.projectaon.org
> User-Agent: curl/7.64.0
> Accept: */*
> 
< HTTP/1.1 404 Not Found
< Date: Sun, 03 Mar 2019 12:43:29 GMT
< Server: Apache
< Expires: Tue, 01 Jan 2002 00:00:00 GMT
< Cache-Control: no-cache
< Set-Cookie: imstime=1551617009; expires=Tue, 02-Apr-2019 12:43:29 GMT; Max-Age=2592000; path=/
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< X-Xss-Protection: 1; mode=block
< X-Frame-Options: SAMEORIGIN
< X-Content-Type-Options: nosniff
< Last-Modified: Sun, 03 Mar 2019 12:43:29 GMT
< Transfer-Encoding: chunked
< Content-Type: text/html; charset=UTF-8
< 
<!DOCTYPE html>
<html lang="en">
  <head>
    <title>Project Aon: Errors / 404 (browse)</title>
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/css/bootstrap.min.css" type="text/css" />
    <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.3/jquery.min.js"></script>
    <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/js/bootstrap.min.js"></script>
    <!--HTMLHeader--><meta http-equiv='Content-Type' content='text/html; charset=utf-8' />        <link rel="alternate" type="application/rss+xml" href="https://www.projectaon.org/en/blog/category/whats-new/feed/" title="What's New - Kai Grand Sentinel" />
<script type="text/javascript">
function toggle(obj, hide, show) {
    var elstyle = document.getElementById(obj).style;
    var text    = document.getElementById(obj + "tog");
    if(!hide) { var hide = "hide"; }
    if(!show) { var show = "show"; }
    if (elstyle.display == 'none') {
        elstyle.display = 'block';
        text.innerHTML  = hide;
    } else {
        elstyle.display = 'none';
        text.innerHTML  = show;
    }
}
</script>    <link rel="stylesheet" href="https://www.projectaon.org/en/pub/skins/ishir/styles/main.css" type="text/css" />
    <link rel="stylesheet" href="https://www.projectaon.org/en/pub/skins/ishir/styles/wiki.css" type="text/css" />
    <link rel="shortcut icon" href="/favicon.ico" />

  </head>
  <body>
    <div class="container">
      <header id="main-header">
        <h1><a href="https://www.projectaon.org/en"><span>Project Aon</span></a></h1>
      </header>

      <nav id="main-navbar" class="navbar navbar-ishir">
          <button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#main-navbar-collapse">
            <span class="icon-bar"></span>
            <span class="icon-bar"></span>
            <span class="icon-bar"></span>
          </button>
        <div class="collapse navbar-collapse" id="main-navbar-collapse">
          <ul class='nav navbar-nav'><li> <a href='https://www.projectaon.org/en/Main/Home'>Home</a>
</li><li><a href='https://www.projectaon.org/en/Main/Books'>Books</a>
</li><li><a href='https://www.projectaon.org/en/Main/WhatsNew'>News</a>
</li><li><a href='https://www.projectaon.org/en/Main/FAQ'>FAQ</a>
</li><li><a href='http://projectaon.proboards.com/' rel='nofollow'>Forum</a>
</li><li><a href='https://www.projectaon.org/en/Main/Links'>Links</a>
</li><li><a href='https://www.projectaon.org/en/Main/HelpUs'>Help Us</a>
</li></ul>
 
        </div>
      </nav>

      <article>
         <!--PageText-->
<div id='wikitext'>
<h2>404 - Not Found</h2>
<p>The requested URL was not found on this server. If the link that you followed was on the Project Aon site, please <a href='https://www.projectaon.org/en/Main/Contact'>contact us</a> so that we can correct the error. We apologize for the annoyance.
</p>
<p>Your life and quest continue <a href='http://www.projectaon.org/' rel='nofollow'>here</a>.
</p>
</div>

      </article>

      <footer>
       <div id="footer-top" class="row">
         <div id="page-info" class="col-sm-5">
           <form id="search-form" class="form-inline" role="form" action="https://www.projectaon.org/en">
             <input type="hidden" name="n" value="Errors.404" />
             <input type="hidden" name="group" value="Errors" />
             <input type="hidden" name="action" value="search" />
             <input type="text" name="q" class="searchbox form-control" />
             <input type="submit" class="btn" value="Search" />
           </form>
           <p>Updated: March 20, 2006, at 11:03 AM</p>
         </div>
         <div class="col-sm-3">
           <p><a href="https://www.projectaon.org/en//Main/Contact">Contact Us</a></p>
         </div>
         <div id="language-selection" class="col-sm-4">
           <p class='list-group'>  <a class='self-language-link list-group-item' href='https://www.projectaon.org/en/' rel='nofollow'>English</a>
 <a class='list-group-item' href='https://www.projectaon.org/es/' rel='nofollow'>Español</a>
 <a class='list-group-item' href='https://www.projectaon.org/it/' rel='nofollow'>Italiano</a>
</p>
 
         </div>
       </div>
       <nav id="page-actions" class="navbar navbar-ishir">
        <button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#page-actions-navbar-collapse">
          <span class="icon-bar"></span>
          <span class="icon-bar"></span>
          <span class="icon-bar"></span>
        </button>
        <div class="collapse navbar-collapse" id="page-actions-navbar-collapse">
         <ul class='nav navbar-nav'><li>       <a accesskey=''  rel='nofollow'  class='selflink' href='https://www.projectaon.org/en/Errors/404'>View</a>
</li><li>      <a accesskey='e'  rel='nofollow'  href='https://www.projectaon.org/en/Errors/404?action=edit'>Edit</a>
</li><li>   <a accesskey='h'  rel='nofollow'  href='https://www.projectaon.org/en/Errors/404?action=diff'>History</a>
</li><li>     <a accesskey=''  rel='nofollow'  href='https://www.projectaon.org/en/Errors/404?action=print'>Print</a>
</li></ul>

        </div>
       </nav>
      </footer>
    </div>
  </body> 
</html>
* Connection #0 to host www.projectaon.org left intact

Offline

#5 2019-03-03 12:49:30

Lone_Wolf
Administrator
From: Netherlands, Europe
Registered: 2005-10-04
Posts: 12,930

Re: [Solved] makepkg/curl unable to download file

loqs wrote:
curl -v https://www.projectaon.org/staff/david/Seventh%20Sense%20Source.zip

fails, see [1]
curl -L also fails as loqs posted



[1]

$ curl -v https://www.projectaon.org/staff/david/Seventh%20Sense%20Source.zip
*   Trying 69.163.153.189...
* TCP_NODELAY set
* Connected to www.projectaon.org (69.163.153.189) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=projectaon.org
*  start date: Feb  7 12:02:13 2019 GMT
*  expire date: May  8 12:02:13 2019 GMT
*  subjectAltName: host "www.projectaon.org" matched cert's "www.projectaon.org"
*  issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
*  SSL certificate verify ok.
> GET /staff/david/Seventh%20Sense%20Source.zip HTTP/1.1
> Host: www.projectaon.org
> User-Agent: curl/7.64.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Date: Sun, 03 Mar 2019 12:41:39 GMT
< Server: Apache
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< X-Xss-Protection: 1; mode=block
< X-Frame-Options: SAMEORIGIN
< X-Content-Type-Options: nosniff
< Last-Modified: Mon, 26 Oct 2015 14:25:23 GMT
< ETag: "33343bc-52302bb4f5ac0"
< Accept-Ranges: bytes
< Content-Length: 53691324
< Content-Type: application/zip
< 
Warning: Binary output can mess up your terminal. Use "--output -" to tell 
Warning: curl to output it to your terminal anyway, or consider "--output 
Warning: <FILE>" to save to a file.
* Failed writing body (0 != 16384)
* Closing connection 0
$

Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.

clean chroot building not flexible enough ?
Try clean chroot manager by graysky

Offline

#6 2019-03-03 12:59:59

schard
Forum Moderator
From: Hannover
Registered: 2016-05-06
Posts: 2,111
Website

Re: [Solved] makepkg/curl unable to download file

@loqs True. It seems like curl chokes on the spaces in the redirection URL.

This does not work:

curl 'https://www.projectaon.org/staff/david/Seventh Sense Source.zip'

While this works:

curl https://www.projectaon.org/staff/david/Seventh%20Sense%20Source.zip

Last edited by schard (2019-03-03 13:01:08)

Inofficial first vice president of the Rust Evangelism Strike Force

Offline

#7 2019-03-03 13:03:01

schard
Forum Moderator
From: Hannover
Registered: 2016-05-06
Posts: 2,111
Website

Re: [Solved] makepkg/curl unable to download file

@LoneWolf
curl won't write binary data to STDOUT.
You need to specify -o <file>.

Inofficial first vice president of the Rust Evangelism Strike Force

Offline

#8 2019-03-03 13:23:00

Lone_Wolf
Administrator
From: Netherlands, Europe
Registered: 2005-10-04
Posts: 12,930

Re: [Solved] makepkg/curl unable to download file

$ curl -L -o test.zip 'https://www.projectaon.org/staff/david/Seventh Sense Source.zip'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  5279    0  5279    0     0   7926      0 --:--:-- --:--:-- --:--:--  7926
$ curl -L -o test1.zip https://www.projectaon.org/staff/david/Seventh%20Sense%20Source.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 51.2M  100 51.2M    0     0  2739k      0  0:00:19  0:00:19 --:--:-- 4025k
[ ls -l
total 52448
-rw-r--r-- 1 panoramix panoramix 53691324  3 mrt 14:20 test1.zip
-rw-r--r-- 1 panoramix panoramix     5279  3 mrt 14:19 test.zip
$

That clearly shows the spaces are the problem.
Looks like a bug in curl wrt redirect processing ?

Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.

clean chroot building not flexible enough ?
Try clean chroot manager by graysky

Offline

#9 2019-03-03 15:06:57

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 30,330
Website

Re: [Solved] makepkg/curl unable to download file

It could be a feature request to curl to accomodate non-compliant servers, but it is a bug in the server putting spaces in a URI.  Spaces are not valid in URIs.  It is suggested that client programs that accept "user typed" URIs should attempt to properly strip or encode spaces (so even this advice would apply to the server) but as spaces are invalid in URI's, curl's behavior is properly following the standard.

https://tools.ietf.org/html/rfc3986

Lone_Wolf wrote:

I'm not sure whether the problem is caused by  an upstream filename with spaces, the redirects or something else.

The upstream file name can certainly have spaces and be served over the web.  But it is the job of the web server to generate a proper URI for this resource in HTTP headers.  So the problem is in the web server that generates an improper redirect (or if the redirect is generated by a PHP 'header' function or similar user script, then the error is in the redirection script not the server software itself.  PHP's 'header' function will not do any uri encoding of a Location header, the author of the script must handle this.)

Last edited by Trilby (2019-03-03 15:33:11)

"UNIX is simple and coherent" - Dennis Ritchie; "GNU's Not Unix" - Richard Stallman

Offline

#10 2019-03-03 20:07:50

eschwartz
Fellow
Registered: 2014-08-08
Posts: 4,097

Re: [Solved] makepkg/curl unable to download file

makepkg's DLAGENTS already uses a perfectly suitable -L in it.

$ curl -I "http://www.projectaon.org/staff/david/Seventh Sense Source.zip"
HTTP/1.1 301 Moved Permanently
Date: Sun, 03 Mar 2019 20:06:17 GMT
Server: Apache
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Xss-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nosniff
Location: https://www.projectaon.org/staff/david/Seventh
Vary: Accept-Encoding
Connection: close
Content-Type: text/html; charset=iso-8859-1

WTF is that Location. Don't blame this on curl -- curl works fine for any other website that uses spaces in filenames. It's hardly an untested field of human endeavor.

Last edited by eschwartz (2019-03-03 20:09:55)

Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#11 2019-03-05 12:51:29

Lone_Wolf
Administrator
From: Netherlands, Europe
Registered: 2005-10-04
Posts: 12,930

Re: [Solved] makepkg/curl unable to download file

It's now clear that the problem is not in the filename but in the location url .

RFC 3986 does talk about percent enconding / decoding but if/when SP <> %20  conversion is necessary eludes me.
I'll accept Trilby's assesment that spaces aren't allowed in URIs .

That indicates curl is doing the right thing and things are not setup correctly at serverside.
Curl could have a clearer error message but i doubt it's worth the effort to create a feature request for that.

I'll adjust the PKGBUILD to use wget as custom download agent.

Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.

clean chroot building not flexible enough ?
Try clean chroot manager by graysky

Offline

#12 2019-03-05 13:25:22

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 30,330
Website

Re: [Solved] makepkg/curl unable to download file

Why not just use the proper url to the zip rather than relying on the server for several redirects?

As shown in this thread, curl will get the proper zip file when you use the https percent-encoded URI.

"UNIX is simple and coherent" - Dennis Ritchie; "GNU's Not Unix" - Richard Stallman

Offline

#13 2019-03-05 13:46:37

Lone_Wolf
Administrator
From: Netherlands, Europe
Registered: 2005-10-04
Posts: 12,930

Re: [Solved] makepkg/curl unable to download file

Because I was sure I had tried that already without succes.
tested and it does indeed work that way, but only with the direct uri.


What's better :
- custom download agent using the upstream provided download link
- use non-published direct URI so default makepkg download agent can handle it ?

Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.

clean chroot building not flexible enough ?
Try clean chroot manager by graysky

Offline

#14 2019-03-05 13:59:18

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 30,330
Website

Re: [Solved] makepkg/curl unable to download file

Perhaps that's a bit subjective.  Assuming the direct URL isn't changing, then I'd strongly favor the latter - why require an additional dependency on wget when it's not need?  However, if the direct URL does regularly change then using the custom agent may be needed.

"UNIX is simple and coherent" - Dennis Ritchie; "GNU's Not Unix" - Richard Stallman

Offline

#15 2019-03-09 13:29:54

Lone_Wolf
Administrator
From: Netherlands, Europe
Registered: 2005-10-04
Posts: 12,930

Re: [Solved] makepkg/curl unable to download file

Thanks, the direct url doesn't appear to have changed in years so i setlled for using that.
uploaded corrected PKGBUILD to aur.

Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.

clean chroot building not flexible enough ?
Try clean chroot manager by graysky

Offline

Pages: 1

Board footer

Powered by FluxBB