Does nodetool cleanup affect Apache Spark rdd.count() of a Cassandra table? -

-

i've been tracking growth of big cassandra tables using spark rdd.count(). 'till expected behavior consistent, number of rows growing.

today ran nodetool cleanup on 1 of seeds , usual ran 50+ minutes.

and rdd.count() returns 1 third of rows did before....

did destroy data using nodetool cleanup? or spark count unreliable , counting ghost keys? got no errors during cleanup , lots don't show out of usual. did seem successful operation, until now.

update 2016-11-13

turns out cassandra documentation set me loss of 25+ million rows of data.

the documentation explicit:

use nodetool status verify node bootstrapped and all other nodes (un) , not in other state. after new nodes running, run nodetool cleanup on each of existing nodes remove keys no longer belong nodes. wait cleanup complete on 1 node before running nodetool cleanup on next node.

cleanup can safely postponed low-usage hours.

well check status of other nodes via nodetool status , , normal (un), here's catch, also need run command nodetool describecluster might find schemas not synced.

my schemas not synced , ran cleanup, when nodes un, , running per documentation. cassandra documentation not mention nodetool describecluster after adding new nodes.

so merrily added nodes, waited till un (up / normal) , ran cleanup.

as result, 25+ million rows of data gone. hope helps others avoid dangerous pitfall. datastax documentation sets destroy data recommending cleanup step of process of adding new nodes.

in opinion, cleanup step should taken out of new node procedure documentation altogether. should mentioned, elsewhere, cleanup practice not in same section adding new nodes...it's recommending rm -rf / 1 of steps virus removal. sure remove virus...

thank aravind r. yarram reply, came same conclusion reply , came here update this. appreciate feedback.

i guessing might have either added/removed nodes cluster or decreased replication factor before running nodetool cleanup. until run cleanup, guess cassandra still reports old key ranges part of rdd.count() old data still exists on nodes.

reference: https://docs.datastax.com/en/cassandra/2.0/cassandra/tools/toolscleanup.html


Comments

Post a Comment

Popular posts from this blog

php - How to display all orders for a single product showing the most recent first? Woocommerce -

-
i have password protected page on frontend. on page, want display full list of purchases associated single woocommerce product id. option 1: i've looked through woocommerce docs , , found: wc_cli_order , list_( $args, $assoc_args ) function lists orders. presumably, orders can filtered product id [--=] filter orders based on order property. line item fields (numeric array, started index zero): line_items.0.product_id option 2: i found article , based on customer id. i've modified code based on guesses meta_key , meta_value. $customer_orders = get_posts( array( 'numberposts' => -1, 'meta_key' => '_product', 'meta_value' => get_product_id(), 'post_type' => wc_get_order_types(), 'post_status' => array_keys( wc_get_order_statuses() ), ) ); how display orders single product showing recent order first? here solution put based on these articles: http://www.w...
Read more

asp.net - How to correctly use QUERY_STRING in ISAPI rewrite? -

-
i'm having difficult time , hope guys can help. using helicon isapi rewrite version 3.1.0.104 on our iis server. edit http.conf file , have tried , still fail. this trying do: redirect url: https://www.domain.com/switch-by-version?version=2.8.5.2594 to: https://test.domain.com/load/load.aspx?tver=2.8.5.2594 the version number @ end source url change, , need target url have same version number @ end of url example above shows. i tried following not work: rewritecond %{query_string} ^version=(\d\d?)\.(\d\d?)\.(\d\d?)\.(.*)$ [nc] rewriterule ^/switch-by-version(.*)$ https://test.domain.com/load/load.aspx?tver=%1 [r=307,nc,l] any appreciated! thanks. if it's running in equivalent context of .htaccess , rule shouldn't start / . here's simpler version: rewritecond %{query_string} ^version=(\d+\.\d+\.\d+(?:\..+)?)$ [nc] rewriterule ^switch-by-version/?$ https://test.domain.com/load/load.aspx?tver=%1 [r=307,nc,l]
Read more

angularjs - How restrict admin panel using in backend laravel and admin panel on angular? -

-
https://scotch.io/tutorials/token-based-authentication-for-angularjs-and-laravel-apps following tutorials, try write authentication restrict admin panel, didn't understood how can that. this admin panel, admin panel don't want show , request database unauthenticated users. my laravel-routes route::get('/admin',function(){ return view('index'); }); route::get('/api/v1/employees/{id?}', 'employees@index'); route::get('/api/v1/users/{id?}', 'employees@users'); route::post('/api/v1/employees', 'employees@store'); route::post('/api/v1/employees/{id}', 'employees@update'); route::delete('/api/v1/employees/{id}', 'employees@destroy'); route::group(['prefix' => 'api'], function() { route::resource('authenticate', 'authenticatecontroller', ['only' => ['index']]); route::post('authenticate', 'authentic...
Read more