web scraping - Getting 'global name not defined' error in Python using scrapy -

-

i've been learning scrapy book called web scraping python ryan mitchell. there's code in book gets external links website. though i'm using same code in book (the thing did changing 'urllib.request' 'urllib2'), keep getting same error. python version 2.7.12. error: 

file "test.py", line 28, in <module> getallexternallinks("http://www.oreilly.com") file "test.py", line 16, in getallexternallinks internallinks = getinternallinks(bsobj, splitaddress(siteurl)[0]) nameerror: global name 'getinternallinks' not defined

this code i'm using. 

from urllib2 import urlopen urlparse import urlparse bs4 import beautifulsoup import re allextlinks = set()  allintlinks = set()  def getallexternallinks(siteurl):      html = urlopen(siteurl)      bsobj = beautifulsoup(html)      internallinks = getinternallinks(bsobj,splitaddress(siteurl)[0])      externallinks = getexternallinks(bsobj,splitaddress(siteurl)[0])      link in externallinks:          if link not in allextlinks:              allextlinks.add(link)              print(link)      link in internallinks:          if link not in allintlinks:              print("about link: "+link)              allintlinks.add(link)              getallexternallinks(link)  getallexternallinks("http://www.oreilly.com")

read example code, before compile it. look, there no getinternallinks() function in code.


Comments

Post a Comment

Popular posts from this blog

php - How to display all orders for a single product showing the most recent first? Woocommerce -

-
i have password protected page on frontend. on page, want display full list of purchases associated single woocommerce product id. option 1: i've looked through woocommerce docs , , found: wc_cli_order , list_( $args, $assoc_args ) function lists orders. presumably, orders can filtered product id [--=] filter orders based on order property. line item fields (numeric array, started index zero): line_items.0.product_id option 2: i found article , based on customer id. i've modified code based on guesses meta_key , meta_value. $customer_orders = get_posts( array( 'numberposts' => -1, 'meta_key' => '_product', 'meta_value' => get_product_id(), 'post_type' => wc_get_order_types(), 'post_status' => array_keys( wc_get_order_statuses() ), ) ); how display orders single product showing recent order first? here solution put based on these articles: http://www.w...
Read more

asp.net - How to correctly use QUERY_STRING in ISAPI rewrite? -

-
i'm having difficult time , hope guys can help. using helicon isapi rewrite version 3.1.0.104 on our iis server. edit http.conf file , have tried , still fail. this trying do: redirect url: https://www.domain.com/switch-by-version?version=2.8.5.2594 to: https://test.domain.com/load/load.aspx?tver=2.8.5.2594 the version number @ end source url change, , need target url have same version number @ end of url example above shows. i tried following not work: rewritecond %{query_string} ^version=(\d\d?)\.(\d\d?)\.(\d\d?)\.(.*)$ [nc] rewriterule ^/switch-by-version(.*)$ https://test.domain.com/load/load.aspx?tver=%1 [r=307,nc,l] any appreciated! thanks. if it's running in equivalent context of .htaccess , rule shouldn't start / . here's simpler version: rewritecond %{query_string} ^version=(\d+\.\d+\.\d+(?:\..+)?)$ [nc] rewriterule ^switch-by-version/?$ https://test.domain.com/load/load.aspx?tver=%1 [r=307,nc,l]
Read more

angularjs - How restrict admin panel using in backend laravel and admin panel on angular? -

-
https://scotch.io/tutorials/token-based-authentication-for-angularjs-and-laravel-apps following tutorials, try write authentication restrict admin panel, didn't understood how can that. this admin panel, admin panel don't want show , request database unauthenticated users. my laravel-routes route::get('/admin',function(){ return view('index'); }); route::get('/api/v1/employees/{id?}', 'employees@index'); route::get('/api/v1/users/{id?}', 'employees@users'); route::post('/api/v1/employees', 'employees@store'); route::post('/api/v1/employees/{id}', 'employees@update'); route::delete('/api/v1/employees/{id}', 'employees@destroy'); route::group(['prefix' => 'api'], function() { route::resource('authenticate', 'authenticatecontroller', ['only' => ['index']]); route::post('authenticate', 'authentic...
Read more