python - How to use a CSV as input data for a Tensorflow neural network? -

i'm attempting write neural network changing mnist ml beginners code. have csv that's organized this:

image_name |nevus? |dysplastic nevus?| melanoma? asdfgjkgdsl.png |1 |0 |0

an image name, , it's one-hot result. each image 1022 x 767, , i'd use color of each pixel input well. such, changed mnist code have 2,351,622 inputs (1022 pixels wide * 767 pixels high * 3 colors per pixel) , 3 outputs.

# tensorflow.examples.tutorials.mnist import input_data # mnist = input_data.read_data_sets("mnist_data/", one_hot=true)  def main():     x = tf.placeholder(tf.float32, [none, 2351622])     w = tf.variable(tf.zeroes([2351622, 3]))     b = tf.variable(tf.zeroes([3]))      y = tf.nn.softmax(tf.matmul(x, w) + b)      y_ = tf.placeholder(tf.float32, [none, 3])     cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y), reduction_indices=[1]))     train_step = tf.train.gradientdescentoptimizer(0.5).minimize(cross_entropy)      init = tf.initialize_all_variables()     sess = tf.session()     sess.run(init)      in range(1000):     example, label = sess.run([features, col5])         # batch_xs, batch_ys = mnist.train.next_batch(100)         # sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})      correct_prediction = tf.equal(tf.argmax(y,1), tf.argmax(y_,1))     accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))     print(sess.run(accuracy, feed_dict={x: mnist.test.images, y_: mnist.test.labels})) 

the commented lines ones have replace data loaded neural network. easiest way 2.3m inputs each image (that i've found) to:

from pil import image import numpy np  list(np.array(image.open('asdfgjkgdsl.png')).ravel().flatten()) 

how can load dataset tensorflow used training neural network?

probably recommended way prepare series of tf_records files. there example in mnist doing that. then, create queue. save space, it's best keep input in png format , use decode_png @ runtime.

in short, first convert (you should write multiple files):

def _bytes_feature(value):     return tf.train.feature(bytes_list=tf.train.byteslist(value=[value]))  def convert():     writer = tf.python_io.tfrecordwriter(output_filename)     filename, nv, dnv, mn in parse_csv(...):        fs = {}        png_data = read_image_as_np_array(filename)        image_name = 'data/image/png'        fs['png_data'] = _bytes_feature(png_data)        fs['label'] = _bytes_feature([nv, dnv, mn])        example = tf.train.example(features=tf.train.features(feature=fs))        writer.write(example.serializetostring())     writer.close() 

then, read it: (put in queue)

reader = tf.tfrecordreader() _, serialized_example = reader.read(filename_from_queue) features_def = {   'png_data': tf.fixedlenfeature([], tf.string),   'label': tf.fixedlenfeature([3], tf.uint8) } features = tf.parse_single_example(serialized_example, features=feature_def) image = tf.image.decode_png(features['png_data']) ... 

you can use tf.textlinereader read line line instead.