Qfyilyi

Normally I can always find my answers on here, but this one seems to pop up here and there, but never gets resolved so here I go:

I was programming a basic Neural Network to be trained with Stochastic Gradient Descent, nothing fancy, most of it copy pasted from a tutorial to get the hang of it and go from there, just wanted to try it out. I recently watched a video about the Google Sheets API, so I wanted to show some stats on a spreadsheet.

Now to the actual problem: Somehow, toExcel() executes twice, although its only called once

Here is my code (python 3.6.7):

import numpy as np

import random

import json

import loader as l

import os

import time

import math

import pickle

import datetime

import psutil



import gspread

from oauth2client.service_account import ServiceAccountCredentials



offset = [1, 7]



def loadNet(filename):

    with open(savedir+filename+".pkl", "rb") as input:

        net = pickle.load(input)

        return net



class Network:



    def __init__(self, layers):

        self.num_layers = len(layers)

        self.layers = layers

        self.biases = [np.random.randn(y, 1) for y in layers[1:]]

        self.weights = [np.random.randn(y, x) for x, y in zip(layers[:-1], layers[1:])]



    def toExcel(self):

        print("Function start")

        scope = ['https://spreadsheets.google.com/feeds','https://www.googleapis.com/auth/drive']

        creds = ServiceAccountCredentials.from_json_keyfile_name('auth_creds.json', scope)

        client = gspread.authorize(creds)



        sheet = client.open("nn_temp").sheet1

        input = 0

        x = 0

        s = time.time()

        for var in range(len(self.layers)):

            if var == 0:

                continue

            index = var-1

            sheet.update_cell(offset[1],offset[0]+x, self.layers[index+1])

            w_cells = sheet.range(offset[1]+1,offset[0]+x,offset[1]+len(self.weights[index]),offset[0]+x)

            for weight in range(len(self.weights[index])):

                w_cells[weight].value = sum(self.weights[index][weight])/float(len(self.weights[index][weight]))

            sheet.update_cells(w_cells)

            b_cells = sheet.range(offset[1]+1,offset[0]+x+1,offset[1]+len(self.biases[index]),offset[0]+x+1)

            for bias in range(len(self.biases[index])):

                b_cells[bias].value = sum(self.biases[index][bias])/float(len(self.biases[index][bias]))

            sheet.update_cells(b_cells)

            del w_cells, b_cells

            x += 2

        print("time: "+str(time.time()-s))

        del sheet,input,x,s,scope,creds,client

        return





net = loadNet("93_22")

print("between") #not in the first result, i added this later

net.toExcel()

All I am doing with loadNet("93_22") is loading a previously saved Network, having specific weights and biases

Console output:

paul@Paul-2:~/proj/dl/stgd$ python3 net.py

Function start

time: 5.690556049346924

Function start

time: 5.826641082763672

EDIT 1: I tried what @Silmathoron proposed and got this:

paul@Paul-2:~/proj/dl/stgd$ python3 net.py

between

Function start

time: 7.506024122238159

between

Function start

time: 6.050253868103027

EDIT 2: pickle is the problem, i tried this:

net = Network([784, 20, 20, 20, 10])

print("between")

net.toExcel()

and got a clean execution

between

Function start

time: 4.296467065811157

guess im gonna use google sheets to save my networks not too