39158

Change max-job size of beanstalkd

Question:

Does somebody how to change the max-job size of beanstalkd ? I have the problem that I get the message <strong>JOB_TOO_BIG</strong> and at <a href="https://stackoverflow.com/questions/24428837/adding-job-to-beanstalkd" rel="nofollow">Adding Job to beanstalkd</a>, it says that the default size is 65k. Does somebody know how to change that ?

EDIT: my beanstalkd init-script in the folder /etc/init.d looks like the following (I added the -z option to increase the job size):

#!/bin/sh # # Copyright (c) 2007 Javier Fernandez-Sanguino <jfs@debian.org> # # This is free software; you may redistribute it and/or modify # it under the terms of the GNU General Public License as # published by the Free Software Foundation; either version 2, # or (at your option) any later version. # # This is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License with # the Debian operating system, in /usr/share/common-licenses/GPL; if # not, write to the Free Software Foundation, Inc., 59 Temple Place, # Suite 330, Boston, MA 02111-1307 USA # ### BEGIN INIT INFO # Provides: beanstalkd # Required-Start: $remote_fs $network $local_fs # Required-Stop: $remote_fs $network $local_fs # Should-Start: $named # Should-Stop: # Default-Start: 2 3 4 5 # Default-Stop: 0 1 6 # Short-Description: in-memory workqueue service # Description: beanstalk is a simple, fast, queueing server. Its # interface is generic, but was originally designed # for reducing the latency of page views in high-volume # web applications by running time-consuming tasks # asynchronously. ### END INIT INFO PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin DAEMON=/usr/bin/beanstalkd # Introduce the server's location here NAME=beanstalkd # Introduce the short server's name here DESC="in-memory queueing server" # Introduce a short description here LOGDIR=/var/log/beanstalkd # Log directory to use BEANSTALKD_LISTEN_ADDR=0.0.0.0 BEANSTALKD_LISTEN_PORT=11300 PIDFILE=/var/run/$NAME.pid test -x $DAEMON || exit 0 . /lib/lsb/init-functions # Default options, these can be overriden by the information # at /etc/default/$NAME DAEMON_OPTS="-l $BEANSTALKD_LISTEN_ADDR -p $BEANSTALKD_LISTEN_PORT -z 550000000000" # Additional options given to the server #DIETIME=10 # Time to wait for the server to die, in seconds # If this value is set too low you might not # let some servers to die gracefully and # 'restart' will not work STARTTIME=1 # Time to wait for the server to start, in seconds # If this value is set each time the server is # started (on start or restart) the script will # stall to try to determine if it is running # If it is not set and the server takes time # to setup a pid file the log message might # be a false positive (says it did not start # when it actually did) #LOGFILE=$LOGDIR/$NAME.log # Server logfile DAEMONUSER=root #beanstalkd # Users to run the daemons as. If this value # is set start-stop-daemon will chuid the server # Include defaults if available if [ -f /etc/default/$NAME ] ; then . /etc/default/$NAME fi # Check that the user exists (if we set a user) # Does the user exist? if [ -n "$DAEMONUSER" ] ; then if getent passwd | grep -q "^$DAEMONUSER:"; then # Obtain the uid and gid DAEMONUID=`getent passwd |grep "^$DAEMONUSER:" | awk -F : '{print $3}'` DAEMONGID=`getent passwd |grep "^$DAEMONUSER:" | awk -F : '{print $4}'` else log_failure_msg "The user $DAEMONUSER, required to run $NAME does not exist." exit 0 fi fi set -e running_pid() { # Check if a given process pid's cmdline matches a given name pid=$1 name=$2 [ -z "$pid" ] && return 1 [ ! -d /proc/$pid ] && return 1 cmd=`cat /proc/$pid/cmdline | tr "\000" "\n"|head -n 1 |cut -d : -f 1` # Is this the expected server [ "$cmd" != "$name" ] && return 1 return 0 } running() { # Check if the process is running looking at /proc # (works for all users) # No pidfile, probably no daemon present [ ! -f "$PIDFILE" ] && return 1 pid=`cat $PIDFILE` running_pid $pid $DAEMON || return 1 return 0 } start_server() { # Start the process using the wrapper if [ "x$START" != "xyes" -a "x$START" != "xtrue" ]; then echo "" echo "beanstalkd not configured to start, please edit /etc/default/beanstalkd to enable" exit 0 fi if [ -z "$DAEMONUSER" ] ; then start_daemon -p $PIDFILE $DAEMON $DAEMON_OPTS errcode=$? else # if we are using a daemonuser then change the user id start-stop-daemon --start --quiet --pidfile $PIDFILE \ --chuid $DAEMONUSER --make-pidfile --oknodo \ --background --exec $DAEMON -- $DAEMON_OPTS errcode=$? fi return $errcode } stop_server() { # Stop the process using the wrapper if [ -z "$DAEMONUSER" ] ; then killproc -p $PIDFILE $DAEMON errcode=$? else # if we are using a daemonuser then look for process that match start-stop-daemon --stop --quiet --pidfile $PIDFILE \ --user $DAEMONUSER \ --exec $DAEMON errcode=$? fi rm -f $PIDFILE return $errcode } reload_server() { [ ! -f "$PIDFILE" ] && return 1 pid=pidofproc $PIDFILE # This is the daemon's pid # Send a SIGHUP kill -1 $pid return $? } force_stop() { # Force the process to die killing it manually [ ! -e "$PIDFILE" ] && return if running ; then kill -15 $pid # Is it really dead? sleep "$DIETIME"s if running ; then kill -9 $pid sleep "$DIETIME"s if running ; then echo "Cannot kill $NAME (pid=$pid)!" exit 0 fi fi fi rm -f $PIDFILE } case "$1" in start) log_daemon_msg "Starting $DESC " "$NAME" # Check if it's running first if running ; then log_progress_msg "apparently already running" log_end_msg 0 exit 0 fi if start_server ; then # NOTE: Some servers might die some time after they start, # this code will detect this issue if STARTTIME is set # to a reasonable value [ -n "$STARTTIME" ] && sleep $STARTTIME # Wait some time if running ; then # It's ok, the server started and is running log_end_msg 0 else # It is not running after we did start log_end_msg 1 fi else # Either we could not start it log_end_msg 1 fi ;; stop) log_daemon_msg "Stopping $DESC" "$NAME" if running ; then # Only stop the server if we see it running errcode=0 stop_server || errcode=$? log_end_msg $errcode else # If it's not running don't do anything log_progress_msg "apparently not running" log_end_msg 0 exit 0 fi ;; force-stop) # First try to stop gracefully the program $0 stop if running; then # If it's still running try to kill it more forcefully log_daemon_msg "Stopping (force) $DESC" "$NAME" errcode=0 force_stop || errcode=$? log_end_msg $errcode fi ;; restart|force-reload) log_daemon_msg "Restarting $DESC" "$NAME" errcode=0 stop_server || errcode=$? # Wait some sensible amount, some server need this [ -n "$DIETIME" ] && sleep $DIETIME start_server || errcode=$? [ -n "$STARTTIME" ] && sleep $STARTTIME running || errcode=$? log_end_msg $errcode ;; status) log_daemon_msg "Checking status of $DESC" "$NAME" if running ; then log_progress_msg "running" log_end_msg 0 else log_progress_msg "apparently not running ... " log_end_msg 1 exit 0 fi ;; reload) log_warning_msg "Reloading $NAME daemon: not implemented (use restart)." ;; *) N=/etc/init.d/$NAME echo "Usage: $N {start|stop|force-stop|restart|force-reload|status}" >&2 exit 1 ;; esac exit 0

And my beanstalkd.conf file which is in the folder /etc/default looks like the following:

## Defaults for the beanstalkd init script, /etc/init.d/beanstalkd on ## Debian systems. Append "-b /var/lib/beanstalkd" for persistent ## storage. BEANSTALKD_LISTEN_ADDR=0.0.0.0 BEANSTALKD_LISTEN_PORT=11300 # create the journal path before use !!! BEANSTALKD_JOURNAL_PATH="/var/lib/beanstalkd" DAEMON_OPTS="-l $BEANSTALKD_LISTEN_ADDR -p $BEANSTALKD_LISTEN_PORT -z 550000000000 -b $BEANSTALKD_JOURNAL_PATH -V" ## Uncomment to enable startup during boot. START=yes

best regards,

Answer1:

The answer by <a href="https://stackoverflow.com/users/3874319/slickorange" rel="nofollow">slickorange</a> in the following link might help you: <a href="https://stackoverflow.com/questions/29199302/job-too-big-pheanstalk-what-can-be-done/#answer-32449935" rel="nofollow">JOB_TOO_BIG Pheanstalk - what can be done?</a> <br />

The size of beanstalkd job can be increased by adding the following line (or uncomment the existing BEANSTALKD_EXTRA line and edit it):

BEANSTALKD_EXTRA="-z 524280"

The size is specified in bytes, default size being 65535 bytes. <br /> Restart beanstalkd after making the change:

sudo service beanstalkd restart

Answer2:

On Debian 8.7 x64 editing the settings in /etc/default/beanstalkd works. A bug probably.

Answer3:

I have systemctl in Debian 8 and my config looks slightly different from what is suggested here in other answers to increase the limit. This is what I did:

<ol><li>

Edit the beanstalkd file in:

nano /etc/sysconfig/beanstalkd

</li> <li>

Then increase the limit for MAX_JOB_SIZE from default 65535 to 524280

MAX_JOB_SIZE=-z 524280

</li> <li>

Restart beanstalkd and check the status:

service beanstalkd restart

systemctl status beanstalkd

</li> </ol>

Answer4:

There are settings for the daemon

-b DIR wal directory -f MS fsync at most once every MS milliseconds (use -f0 for "always fsync") -F never fsync (default) -l ADDR listen on address (default is 0.0.0.0) -p PORT listen on port (default is 11300) -u USER become user and group -z BYTES set the maximum job size in bytes (default is 65535) -s BYTES set the size of each wal file (default is 10485760) (will be rounded up to a multiple of 512 bytes) -c compact the binlog (default) -n do not compact the binlog -v show version information -V increase verbosity -h show this help

So based on your Linux, you should find out where this is kept and change it. Usually it's under beanstalkd.conf

Recommend

  • Flask-Bcrypt - AttributeError: 'module' object has no attribute 'ffi' - deployed
  • Check if a program exists in bash
  • How to exit “docker run” containers once the script those containers execute calls exit()
  • update record in database using jdatabase
  • Python adding lots of things to PATH. How do I stop?
  • Simple linked list-C
  • SQLite connection strategies
  • How to calculate the camera position from Vuforia GL matrix?
  • Checking if an array in C is symmetric
  • How do I signal completion of my dataflow?
  • Convert Type Decimal to Hex (string) in .NET 3.5
  • Firefox Extension - Monitor refresh and change of tab
  • Reduction and collapse clauses in OMP have some confusing points
  • Appending Character to Character Array In C
  • ADO and msqli connections very slow
  • Saving Changes After In-App Purchase Has Been Purchased
  • How can the INSERT … ON CONFLICT (id) DO UPDATE… syntax be used with a sequence ID?
  • OOP Javascript - Is “get property” method necessary?
  • Parsing a CSV string while ignoring commas inside the individual columns
  • GridView breaks while scrolling
  • QLPreviewController hide print button in ios6
  • PHP buffered output depending on server setting?
  • copying resource to sdcard gives a damaged file in android
  • Database structure design with variable amounts of fields
  • Avoid links criss cross / overlap in d3.js using force layout
  • how to adjust image in a panel in Java swing?
  • Java Scanner input dilemma. Automatically inputs without allowing user to type
  • Uncaught Error: Could not find module `ember-load-initializers`
  • Problems to linebreak with an int in JLabel
  • How reduce the height of an mschart by breaking up the y-axis
  • output of program is not same as passed argument
  • MySQL WHERE-condition in procedure ignored
  • To display the title for the current loaction in map in iphone
  • Rearranging Cells in UITableView Bug & Saving Changes
  • How get height of the a view with gone visibility and height defined as wrap_content in xml?
  • FormattedException instead of throw new Exception(string.Format(…)) in .NET
  • Are Kotlin's Float, Int etc optimised to built-in types in the JVM? [duplicate]
  • Sorting a 2D array using the second column C++
  • costura.fody for a dll that references another dll
  • java string with new operator and a literal