elasticsearch - Aggregate only newest document -

-

i have elastic index has documents user state history. data looks this;

  {     "session_id": "yunus",     "state_name": "start",     "entry_time": "2016-11-09 15:27:03"   },   {     "session_id": "yunus",     "state_name": "end",     "entry_time": "2016-11-09 16:30:00"   },   {     "session_id": "can",     "state_name": "start",     "entry_time": "2016-11-09 12:01:00"   },   {     "session_id": "rick",     "state_name": "start",     "entry_time": "2016-11-09 09:00:00"   },   {     "session_id": "rick",     "state_name": "end",     "entry_time": "2016-11-10 10:00:00"   }

i want aggregate state name date histogram relevant last state @ time. result can be;

2016-11-08  start = 0 end = 0  2016-11-09  start = 2 end = 1  2016-11-10  start = 1 end = 2

actually plan generate grouped bar chart timeline show states change on time.

i tried several things aggregation pipelines, top hits couldn't make progress.

any appreciated.

for interested, solved spark. used elastic-spark read elasticsearch , write elasticsearch.

here read es rdd;

val alldata = sc.esrdd(s"states_${id}/log", query)

then first group session id, sort date find latest state of session;

val lateststates = alldata.groupby(k => k._2.get("session_id").get).map(k => (k._2).reduceleft((d1, d2) => {   d1._2.get("timestamp").get.asinstanceof[long] > d2._2.get("timestamp").get.asinstanceof[long] match {     case true => d1     case false => d2   } })).map(_._2)

once have latest states of session, filter exit states count value;

val statesummary = lateststates   .filter(s => s.isdefinedat("state_id") && s("state_id").asinstanceof[long] != -1)   .map(s => (s("state_id"), s("state_name")))   .countbyvalue()   .map(d => map("state_id" -> d._1._1.asinstanceof[long], "state_name" -> d._1._2.asinstanceof[string], "count" -> d._2)).tolist

now have current number of sessions in states. (current configurable can set specific time), thing left, write elasticsearch;

sc.makerdd(seq(finalelasticdoc)).savetoes(s"states_${id}/analytic_daily")

Comments

Post a Comment

Popular posts from this blog

php - How to display all orders for a single product showing the most recent first? Woocommerce -

-
i have password protected page on frontend. on page, want display full list of purchases associated single woocommerce product id. option 1: i've looked through woocommerce docs , , found: wc_cli_order , list_( $args, $assoc_args ) function lists orders. presumably, orders can filtered product id [--=] filter orders based on order property. line item fields (numeric array, started index zero): line_items.0.product_id option 2: i found article , based on customer id. i've modified code based on guesses meta_key , meta_value. $customer_orders = get_posts( array( 'numberposts' => -1, 'meta_key' => '_product', 'meta_value' => get_product_id(), 'post_type' => wc_get_order_types(), 'post_status' => array_keys( wc_get_order_statuses() ), ) ); how display orders single product showing recent order first? here solution put based on these articles: http://www.w...
Read more

asp.net - How to correctly use QUERY_STRING in ISAPI rewrite? -

-
i'm having difficult time , hope guys can help. using helicon isapi rewrite version 3.1.0.104 on our iis server. edit http.conf file , have tried , still fail. this trying do: redirect url: https://www.domain.com/switch-by-version?version=2.8.5.2594 to: https://test.domain.com/load/load.aspx?tver=2.8.5.2594 the version number @ end source url change, , need target url have same version number @ end of url example above shows. i tried following not work: rewritecond %{query_string} ^version=(\d\d?)\.(\d\d?)\.(\d\d?)\.(.*)$ [nc] rewriterule ^/switch-by-version(.*)$ https://test.domain.com/load/load.aspx?tver=%1 [r=307,nc,l] any appreciated! thanks. if it's running in equivalent context of .htaccess , rule shouldn't start / . here's simpler version: rewritecond %{query_string} ^version=(\d+\.\d+\.\d+(?:\..+)?)$ [nc] rewriterule ^switch-by-version/?$ https://test.domain.com/load/load.aspx?tver=%1 [r=307,nc,l]
Read more

angularjs - How restrict admin panel using in backend laravel and admin panel on angular? -

-
https://scotch.io/tutorials/token-based-authentication-for-angularjs-and-laravel-apps following tutorials, try write authentication restrict admin panel, didn't understood how can that. this admin panel, admin panel don't want show , request database unauthenticated users. my laravel-routes route::get('/admin',function(){ return view('index'); }); route::get('/api/v1/employees/{id?}', 'employees@index'); route::get('/api/v1/users/{id?}', 'employees@users'); route::post('/api/v1/employees', 'employees@store'); route::post('/api/v1/employees/{id}', 'employees@update'); route::delete('/api/v1/employees/{id}', 'employees@destroy'); route::group(['prefix' => 'api'], function() { route::resource('authenticate', 'authenticatecontroller', ['only' => ['index']]); route::post('authenticate', 'authentic...
Read more