Search Members Help

» Welcome Guest
[ Log In :: Register ]

Mini-ITX Boards Sale, Fanless BareBones Mini-ITX, Bootable 1G DSL USBs, 533MHz Fanless PC <-- SALE $200 each!
Get The Official Damn Small Linux Book. DSL Market , Great VPS hosting provided by Tektonic
Pages: (16) </ ... 2 3 4 5 6 [7] 8 9 10 11 12 ... >/

[ Track this topic :: Email this topic :: Print this topic ]

reply to topic new topic new poll
Topic: MyDSL info parser, prototype< Next Oldest | Next Newest >
mikshaw Offline





Group: Members
Posts: 4856
Joined: July 2004
Posted: Mar. 16 2008,13:06 QUOTE

Quote
I think your collapsable browser would be nicer than those two drop downs.
I thought about that yesterday morning, but at the time I didn't feel it was worth attempting until after making the updatedb function more reliable.


Quote
Xfree86-devel.unc is the only one that screws up.
Just noticed that it contains two "Current:" lines


--------------
http://www.tldp.org/LDP/intro-linux/html/index.html
Back to top
Profile PM WEB 
roberts Offline





Group: Members
Posts: 4983
Joined: Oct. 2003
Posted: Mar. 16 2008,18:31 QUOTE

Quote (mikshaw @ Mar. 16 2008,05:06)
Quote
I think your collapsable browser would be nicer than those two drop downs.
I thought about that yesterday morning, but at the time I didn't feel it was worth attempting until after making the updatedb function more reliable.


Quote
Xfree86-devel.unc is the only one that screws up.
Just noticed that it contains two "Current:" lines

Thanks for the update on XFree86-devel.unc. I have fixed it.

This is that sort of thing that I have been "cleaing up" to try to get data to be INSERTable to an SQL table. It has been more work that I originally thought.

Anyway, I have posted and updated mydslinfo.bz2
Back to top
Profile PM WEB 
mikshaw Offline





Group: Members
Posts: 4856
Joined: July 2004
Posted: Mar. 16 2008,21:37 QUOTE

Quote
This is that sort of thing that I have been "cleaing up" to try to get data to be INSERTable to an SQL table.
I'm guessing that converting all dates to yyyy/mm/dd (if that's in your plan)  is going to be a pain.

Here's a version that uses the collapsible browser rather than two separate menus.

There are a few things to consider....

It hasn't been thoroughly tested. There are probably issues I haven't seen yet.

In order for it to work as is, it needs the database to retain two existing qualities (besides the obvious keywords Location, Title, etc)
1) A line break between every package
2) The database has all apps together, all games together, etc. It currently compares the Location string to an existing variable to decide whether or not to create a submenu. If the categories get mixed in the database file, there will be multiple copies of some submenu headers.

It still needs some tweaks of the keyboard control. For example, after installation or database update, you need to either hit Tab a few times or switch to search and back to browse in order to get keyboard focus to the file list.

I don't yet know a way to change the sort order of the list, which is a feature I've had in the back of my mind since the beginning. I'm hoping to be able to allow the user to list files in alphabetical or chronological order (at least in the search window), and maybe have user-defined columns, like license, author, and maybe description.

I was also considering having only one category expanded at any one time, but for some reason that was giving me a lot more trouble than I had expected.

EDIT: fixed the browser doubling up after a db update.
Also looking into an easier way to keep track of which info table index is required at any given time rather than having to run down through it each time.

Code Sample
#!/home/dsl/bin/murgaLua-0.6.4
--[[
    MyDSL info browser, 2008 mikshaw

    Changelog (yyyy/mm/dd)
      2008/03/16: Interface tweaks
      2008/03/15: Improved the update function
                  Database date check uses os.date/lfs instead of os.popen/ls/awk
                  Use wizard instead of tabs for better use of resources
      2008/03/14: Undid gmatch change due to improvements to the database file
                  Improved keyboard control
                  Use Fl_Menu_Button for menus to add to keyboard control
                  Fixed crash on search or install with an empty database
      2008/01/24: Fixed string.gmatch to break up files even if there is no newline between them
      2008/01/17: Added date of last update
                  Tweaked the update function (it's still sloppy)
                  Fixed ".mydsl_dir not found" error message
      2008/01/16: Fixed resizable behavior
                  Uses file in .mydsl_dir
                  Added auto update for first run
      2008/01/15: Added text search
      2008/01/14: Fixed display of incorrect duplicate info file
                  Check for trailing spaces in title string
                  Added "Update" function
                  Code and interface tweaks
      2008/01/13: start of project
   
--]]

mydsldirfile=io.open("/opt/.mydsl_dir")
if not mydsldirfile then
 print(arg[0]..": Can't open /opt/.mydsl_dir")
 os.exit(1)
end
mydsl_dir=mydsldirfile:read("*l")
mydsldirfile:close()

listfile=mydsl_dir.."/mydslinfo.bz2"
listurl="ftp://ibiblio.org/pub/Linux/distributions/damnsmall/mydsl/mydslinfo.bz2"
list_sep="@b@B50@u"

function toggle(b)
local count,state,one,two=1
-- selected line and its label
local me,label=b:value(),b:text(b:value())
if string.find(label,"%+%s*") then
 state,one,two="expanded","%+  ","%-  "
else
 state,one,two="collapsed","%-  ","%+  "
end
b:text(me,string.gsub(label,one,two)) -- swap + and -
-- toggle all lines until reaching the next section or the browser end
while b:text(me+count) and not string.match(b:text(me+count),list_sep) do
 if state=="collapsed" then b:hide(me+count)
 else b:show(me+count)
 end
 count=count+1
end
end

function fill_list()
inputfile = io.popen("bunzip2 -c "..listfile)
data=inputfile:read("*a")
inputfile:close()
if not string.find(data,"%w") then -- if there's nothing readable in it
 info_display_buffer:text("Cannot read database "..listfile.."\nPlease try updating again.")
else
filelist:clear()
header={}
info={} -- table containing all package data: info[index].{location,title,date}
info_count=0 -- number of packages
local current_loc="" -- location string
local headers=0 -- nuber of locations
-- breaks up info for each package into a separate string
for s in string.gmatch(data,"(Location:.-)\nLocation:") do
 info_count=info_count+1
 info[info_count]={
 location=string.match(s,"Location:%s*(.-)%s*\n"),
 title=string.match(s,"Title:%s*(.-)%s*\n"),
 date=string.match(s,"Current:%s*(.-%d)%s"),
 text=s
 }
 if current_loc ~= info[info_count].location then
   -- create a header when location changes
   current_loc=info[info_count].location
   filelist:add(list_sep.."-  "..current_loc)
   headers=headers+1
   header[headers]=filelist:size()
 end
 filelist:add(info[info_count].title)
end
for i,v in ipairs(header) do filelist:value(header[i]); toggle(filelist) end
last_update:label("last update "..os.date("%d %b %Y",lfs.attributes(listfile).modification))
end
filelist:value(1)
tab_browse:setonly()
end

function updatedb()
-- download to temp file, check file integrity, and only then replace the old file
 tempfile=os.tmpname()
 os.execute("aterm +tr -bg white -fg black -geometry 80x4 -title \"Database Update\" -e wget "..listurl.." -O "..tempfile)
 if os.execute("bzip2 -t "..tempfile) == 0 then
   os.rename(tempfile,listfile)
   fill_list()
 else fltk.fl_alert("Database check failed.\nThis may be the result of an incomplete download\nor gremlins in your internets.")
 os.remove(tempfile)
 end
end

function pick_search_result()
if results:value() > 1 then
-- show selected info file in the browse tab
 get_loc=string.match(results:text(results:value()),spacer.."(.*)"..spacer)
 get_fname=string.match(results:text(results:value()),"(.-)"..spacer)
-- select the package in filelist
 local myloc=1
 while myloc<=filelist:size() do
   if not string.find(filelist:text(myloc),"  "..get_loc,1,1) then
     myloc=myloc+1
   else filelist:value(myloc)
     break
   end
 end
 -- open submenu if not open
 if string.find(filelist:text(myloc),list_sep.."+  ",1,1) then toggle(filelist) end
 while myloc<=filelist:size() do
   if filelist:text(myloc) ~= get_fname then
     myloc=myloc+1
   else filelist:value(myloc)
     break
   end
 end
 for i=1,info_count do
   -- make sure you get the right info AND category
   if info[i].title == get_fname and info[i].location == get_loc then
     info_display_buffer:text(info[i].text)
     tab_browse:setonly()
     myfile=info[i] -- this is the file that will be downloaded
     break
   end
 end
 tabs:value(tab1)
end
end

function list_cb()
if filelist:value()>0 then -- clicking a blank space can be fatal without this check
 if Fl:event() == fltk.FL_RELEASE or Fl:event_key()==fltk.FL_Enter then -- on click or Enter
   if string.find(filelist:text(filelist:value()),list_sep) then
     toggle(filelist) -- if it's a separator
   else
   -- find out what submenu we're under
     local myloc=filelist:value()
     while myloc > 0 do
       myloc=myloc-1
       if string.find(filelist:text(myloc),list_sep) then
         myloc=string.gsub(filelist:text(myloc),".-%s%s","")
         break
       end
     end
     for i=1,info_count do
       -- make sure you get the right info AND category
       if info[i].title == filelist:text(filelist:value()) and info[i].location == myloc then
         info_display_buffer:text(info[i].text)
         myfile=info[i]
         tab_browse:setonly()
         break
       end
     end
   end
 end
end
end

ww=500; wh=360; bh=30; bw=ww/2-5
win=fltk:Fl_Window(ww,wh,"MyDSL Info Browser")

tabs=fltk:Fl_Wizard(0,0,ww,wh-bh-5)
tab1=fltk:Fl_Group(0,0,ww,wh-bh-5,"browse")

tiles=fltk:Fl_Tile(5,5+bh,ww-10,wh-(15+bh*2))
filelist=fltk:Fl_Hold_Browser(5,5+bh,ww/2-5,wh-(15+bh*2))
filelist:callback(list_cb)
info_display=fltk:Fl_Text_Display(ww/2,5+bh,ww/2-5,wh-(15+bh*2))
info_display:textfont(fltk.FL_SCREEN)
info_display:textsize(12)
info_display_buffer=fltk:Fl_Text_Buffer()
info_display:buffer(info_display_buffer)
fltk:Fl_End() --end tiles

-- offscreen button that allows the list to respond to Enter key
enter=fltk:Fl_Return_Button(ww,wh,10,10)
enter:callback(function() filelist:do_callback() end)

install=fltk:Fl_Button(ww/2,5,bw,bh,"&Install Selected Package")
install:callback(
function()
if myfile then
 mydslload=os.execute("mydsl-load "..myfile.title.." "..myfile.location)
 if mydslload~=0 then fltk.fl_alert(myfile.title..":\nDownload or Checksum error!") end
end
end
)

b_updatedb=fltk:Fl_Button(5,5,bw,bh,"&Update Database")
b_updatedb:callback(updatedb)
fltk:Fl_End() --end tab1

tab2=fltk:Fl_Group(0,0,ww,wh-bh-5,"search text")
search_text=fltk:Fl_Input(85,5,ww-90,bh,"search text ")
search_text:when(fltk.FL_WHEN_ENTER_KEY)
search_text:callback(
function()
 if search_text:value() ~= "" and info_count and info_count>0 then
   results:clear()
   if Fl_Browser.column_widths then
     results:add("@B49@b@uPackage Name"..spacer.."@B49@b@uLocation"..spacer.."@B49@b@uRelease Date")
   else
     -- default screen font does not have bold or underline
     -- and no columns means only one format string is used
     results:add("@B49PACKAGE NAME"..spacer.."LOCATION"..spacer.."RELEASE DATE")
   end
   for i=1,info_count do
     if string.find(string.lower(info[i].text),string.lower(search_text:value()),1,true) then
       if not info[i].date then info[i].date="" end
       results:add(info[i].title..spacer..info[i].location..spacer..info[i].date)
     end
   end
 end
end
)

results=fltk:Fl_Hold_Browser(5,5+bh,ww-10,wh-(15+bh*2))
-- column widths available in 0.6.4+
if Fl_Browser.column_widths then
 results:column_widths({results:w()/2,200,0})
 spacer="\t"
else spacer="\t\t" -- force some extra space for older versions
results:textfont(fltk.FL_SCREEN) -- fixed-width makes it closer to columns
end
results:callback(
-- react to click only
function()
if results:value() > 1 and Fl:event()==fltk.FL_RELEASE then
 pick_search_result() end
end
)
-- offscreen results_callback button for Enter key
results_cb=fltk:Fl_Return_Button(ww,wh,bh,bh)
results_cb:callback(pick_search_result)
fltk:Fl_End() --end tab2
fltk:Fl_End() --end tabs group

-- using buttons instead of tabs gives better keyboard control
tabbuttons=fltk:Fl_Pack(5,wh-bh,200,bh-5)
tabbuttons:type(fltk.FL_HORIZONTAL)
tabbuttons:spacing(5)
tab_browse=fltk:Fl_Radio_Button(0,0,100,bh-5,"&browse")
tab_browse:when(fltk.FL_WHEN_CHANGED)
tab_browse:box(fltk.FL_BORDER_BOX); tab_browse:selection_color(7)
tab_browse:callback(function() tabs:value(tab1) end)
tab_search=fltk:Fl_Radio_Button(0,0,100,bh-5,"text &search")
tab_search:when(fltk.FL_WHEN_CHANGED)
tab_search:box(fltk.FL_BORDER_BOX); tab_search:selection_color(7)
tab_search:callback(
function()
 tabs:value(tab2)
 Fl:focus(search_text)
end
)
fltk:Fl_End() --end tabs buttons
last_update=fltk:Fl_Box(bw+5,wh-bh-5,bw,bh+5)


win:resizable(tabs)
tabs:resizable(tab1)
tabs:resizable(tab2)
tab1:resizable(tiles)
tab2:resizable(results)
win:show()
find_file=io.open(listfile)
if find_file then data=find_file:read("*a") else data="" end
if string.find(data,"%w") then
 find_file:close()
 fill_list()
else updatedb()
end
Fl:run()


--------------
http://www.tldp.org/LDP/intro-linux/html/index.html
Back to top
Profile PM WEB 
roberts Offline





Group: Members
Posts: 4983
Joined: Oct. 2003
Posted: Mar. 16 2008,22:52 QUOTE

I handled all the different date formats with standard Lua captures while building and inserting to sqlite.

I think it only fair for me to show you mine since we may want to join some features especially if you are wanting to sort etc.

I started by using your GUI and stripping all flatfile code.
I added some "when" events and used "pack" on some layouts.
I stared by creating a simple database ignoring comments and other "multiline/multivalue" fields.
I then wrote standard lua and luasqlite as binding was available in murgaLua to query, display an index of 'hits' and then a view.
Later I moved the GUI around using a menu for update and quit.
Added the "comments" to database. Added QBE screen, still working on it.

This is a work in progress and I post it only as a preview. The next build of the database will break everything.

I post mainly for mikshaw but anyone can preview it. It currently only support direct sql queries. If you don't know sql don't preview it. The QBE is not implemented yet.

Of course you can download the mydsl.db open a shell start sqlite and issue sql commands too. murgaLua does not currently support readline in Linux so expect to do alot of CLI typing.

Open to suggestions on either the SQL database and the supporting GUI.

I am thinking that the multiline change-log and comments need to be separate tables.

I am not asking anyone to finish it as I am currently working on many parts. I start many sections of code before I "button" things up. That's just how I "work" er, play.

Oh, and this works with DSL 4.2.5 or DSL 3.4.11 no need to grab latest murgaLua

Code Sample


#!/bin/murgaLua
--[[
 MyDSL Extension Browser, 2008 mikshaw - SQL and other GUI mods by Robert Shingledecker
 2008/02/26: Added packed comments (probably needs sep table
 2008/02/16: Added menu for Update database and Quit.
 2008/02/14: Created QBE tab based on feedback not finished.
 2008/02/08: Basic SQL queries working - take to Scale6x to show John.
 2008/01/28: Dropped columns and switched to vertical VIEW layout.
 2008/01/24: Created sqlite database mydsl.db with only simple cols.
 2008/01/21: Gutted all flat file logic kept some GUI.
 2008/01/13: Started from mikshaw's 2008 mydsl info browser.
]]--

function updatedb(nofile)
  if not nofile then backup=os.rename(database,database..".bak") end
  os.execute("aterm  +tr -bg white -fg black -geometry 80x4 -title \"Database Update\" -e wget "..dburl.." -O "..database)
end

function doSQL()
if inputSQL:value() ~= "" then
  results:clear()
  selected = {}
  field_names = {}
  stmt = db:prepare(inputSQL:value())
  if stmt == nil then
     fltk.fl_alert(db:errmsg())
  else
     field_names = stmt:get_names()
     for row in stmt:rows(inputSQL:value()) do
        results:add(row[1].."\t"..row[2])
        selected[#selected+1] = row
     end
     stmt:finalize()
  end
end
end

function showSQL()
  if results:value() > 0 then
     viewResults = ""
     for i = 1, #field_names do
        if field_names[i] == "comments" then
           selected[results:value()][i] = string.gsub(selected[results:value()][i],"%^","\n")
        end
        viewResults = viewResults .. field_names[i].."\t"
        viewResults = viewResults .. selected[results:value()][i].."\n"
     end
     info_display:value(viewResults)
     tabs:value(tab1)
     tab1:hide()
     tab3:show()
  end
end


ww=420; wh=360; bh=25
w=fltk:Fl_Window(ww,wh,"MyDSL Extension Browser")
w:callback(
function(w)
 db:close()
 os.exit(0)
end)

mb = fltk:Fl_Menu_Bar(0,0,ww,bh)
mb:add("&File/&UpdateDB",0,updatedb)
mb:add("&File/E&xit",0,quit)
mb:callback(menu_items)

tabs=fltk:Fl_Tabs(0,bh,ww,wh-bh)

tab1=fltk:Fl_Pack(1,bh+1,ww,0,"SQL")
inputSQL=fltk:Fl_Input(0,0,ww-10,bh)
inputSQL:when(fltk.FL_WHEN_ENTER_KEY_ALWAYS)
inputSQL:callback(doSQL)

results=fltk:Fl_Select_Browser(0,0,ww-10,wh-(8+bh*3))
results:textfont(fltk.FL_SCREEN)
results:textsize(12)
results:when(fltk.FL_WHEN_CHANGED)
results:callback(showSQL)
fltk:Fl_End()

tab2=fltk:Fl_Group(0,0,ww,wh-bh,"QBE")
field_label = {"loc","title","desc","ver","author","site","policy","user","current","comments"}
field_len = {120,180,325,240,320,290,325,285,110,335}
field_in = {}
for i = 1, #field_label do
 field_in[i] = fltk:Fl_Input(75,28+(28*(i-1)),field_len[i],bh,field_label[i])
end
okBtn = fltk:Fl_Return_Button(340, 310, 70, 20, "OK");
okBtn:callback(
function(okBtn)
 for i = 1, #field_label do
   if field_in[i]:value() ~= "" then
     print(field_label[i] .." ".. field_in[i]:value())
   end
 end
end)
fltk:Fl_End()

tab3=fltk:Fl_Group(0,0,ww,wh-bh,"View")
info_display=fltk:Fl_Multiline_Output(4,bh+4,ww-10,wh-(30+bh*2))
info_display:textfont(fltk.FL_SCREEN)
info_display:textsize(12)

install=fltk:Fl_Return_Button(340,310,70,20,"Install")
install:callback(
function()
os.execute("mydsl-load "..selected[results:value()][2].." "..selected[results:value()][1])
end
)

w:resizable(tabs)
tabs:resizable(tab1)
tabs:resizable(tab2)
tab1:resizable(info_display)
tab2:resizable(results)
w:show()

mydsldirfile=io.open("/opt/.mydsl_dir")
if not mydsldirfile then
print("can't open "..mydsldirfile)
os.exit(1)
end
mydsl_dir=mydsldirfile:read("*l")
mydsldirfile:close()

database = mydsl_dir.."/mydsl.db"
dburl="ftp://ibiblio.org/pub/Linux/distributions/damnsmall/mydsl/mydsl.db"
databaseCheck = io.open(database)
if databaseCheck then
  databaseCheck:close()
else
  updatedb(1)
end

selected = {}
field_names = {}
db = sqlite3.open(database)
inputSQL:value("select * from mydsl order by loc;")

Fl:run()
Back to top
Profile PM WEB 
mikshaw Offline





Group: Members
Posts: 4856
Joined: July 2004
Posted: Mar. 16 2008,22:58 QUOTE

Thanks for sharing. I'll try it very soon.

Quote
Oh, and this works with DSL 4.2.5 or DSL 3.4.11 no need to grab latest murgaLua
Are you implying there is trouble with my experiment beyond the lack of columns in murgaLua versions prior to 0.6.4? I tried to make sure it works in 0.5.5, but maybe I missed?


--------------
http://www.tldp.org/LDP/intro-linux/html/index.html
Back to top
Profile PM WEB 
76 replies since Jan. 13 2008,15:39 < Next Oldest | Next Newest >

[ Track this topic :: Email this topic :: Print this topic ]

Pages: (16) </ ... 2 3 4 5 6 [7] 8 9 10 11 12 ... >/
reply to topic new topic new poll
Quick Reply: MyDSL info parser

Do you wish to enable your signature for this post?
Do you wish to enable emoticons for this post?
Track this topic
View All Emoticons
View iB Code