django - Celery group multiple tasks in one design -


i getting familiar celery , have question. setup django-redis-celery

lets take example of task sending email:

tasks

@task def send_email(message):    mailserver.sendonemessage(message) 

views

class newaccount(apiview):   def post(self, request, format=none):      send_email.delay(request.data.email) 

this works perfectly, django sends messages redis , picked celery execute task. want improve system celery picks messages redis @ intervals , executes single task multiple messages. because, connecting email server slow , sending multiple messages single request result in faster process.

i want work:

tasks

@task def send_emails(messages):    mailserver.sendmultiplemessages(messages) 

thoughts?

since using redis cache (django-redis) implemented following workflow:

step 1. create task adds new emails cache

@shared_task() def add_email(user_id):    cache.set("email#{}".format(user_id), none, timeout=none) 

step 2. create periodic task runs every second , looks new emails in cache

class processemailstask(periodictask):    run_every = timedelta(seconds=1)     def run(self, **kwargs):        call_email()  def call_email():    item_exists = true    ids = []    while item_exists:     try:         key = next(cache.iter_keys("email#*"))         ids.append(key.split("email#")[1])         cache.delete_pattern(key)     except:         item_exists = false    if len(ids) > 0:         send_emails_to(ids) 

step 3. run both celery workers , celery beat , profit!


Comments

Popular posts from this blog

php - How to display all orders for a single product showing the most recent first? Woocommerce -

asp.net - How to correctly use QUERY_STRING in ISAPI rewrite? -

angularjs - How restrict admin panel using in backend laravel and admin panel on angular? -