Giters

sympmarc / SPServices

SPServices is a jQuery library which abstracts SharePoint's Web Services and makes them easier to use. It also includes functions which use the various Web Service operations to provide more useful (and cool) capabilities. It works entirely client side and requires no server install.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Move documentation from Codeplex to GitHub

sympmarc opened this issue · comments

commented

Codeplex is dying, but literally and figuratively. I'd like to move the documentation from Codeplex to here on GitHub. I think that GitHub Pages seems like the right mechanism to use, and I've created a gh-pages directory to contain the content.

Convert Documentation

  • Create md files for documentation pages
  • Include needed YAML headers in .md files

Specific doc files still to do

General

Core (docs/core/api/)

Copy

Lists

People

SiteData

Users and Groups

Views

WebPartPages

Webs

Spellchecker

UserProfileService

Workflow

Value-Added (docs/value-added/)

Utilities (docs/utilities/)

Generate Documentation

  • Create gulp task to take md files on input and output html
  • Create gulp task to run local dev server to verify html docs in the browser
  • Create gulp task to publish documentation to gh-pages
commented

Yes, pages is the right tool github to display/host the help pages.  I would propose the following:

  •   Under the MASTER  branch, create a new top-level folder called 'docs'. This is where all of your docs should go. I know you named it "gh-pages" right now, but that could get confusing because there is already a branch with that name where the pages actually live. -    Come up with the structure for this folder for organizing the docs.  Similar to what you did with the source.  It could just start with the same structure you have today on CodePlex. Important thing is to create the skeleton files to meet your needs/liking, even if all they have is a tittle.-     On the gh-pages branch, build an SPA that will read the .md files and output HTML.  Or: create a build script (Gruntfile) there that will generate the static HTML files for you. Those pages are then committed to source and show up on the web pages. 
    Once you have the structure in place, you will be set up for the conversion to start.  Others can help (hopefully). 
    As an example, if you take a look at the pages for one of my projects (example), you will see that it actually displays the content of all doc files (.md files) in one page. I currently do that manually  - meaning: I have a grunt task that combines all .md doc files into one, which I then convert to HTML using Dingus and then manually copy the HTML into the index.html page.  Those are steps I'm looking to automate for my own projects, so I'll happy to contribute here as well when I'm done. 

Btw: have you worked directly with the content in the gh-pages branch (ex. Have you checked out that branch into your local workspace)? If not, I can show you. The only thing is that once you start to work with directly, you have to stop using the Github WYSIWYG editor. But thats not a big deal. 

Paul
-- sent from mobile

On Tue, Jun 2, 2015 at 6:15 AM -0700, "Marc D Anderson" notifications@github.com wrote:

Codeplex is dying, but literally and figuratively. I'd like to move the documentation from Codeplex to here on GitHub. I think that GitHub Pages seems like the right mechanism to use, and I've created a gh-pages directory to contain the content.

—
Reply to this email directly or view it on GitHub.

commented

It seems like the gh-pages branch is how GitHub Pages are fed, so I think there's already a process there for doc publishing. I should probably look at some other GitHub Pages -based sites to see how they handle it. I'm going from the docs here to try to get up and running: https://help.github.com/categories/github-pages-basics/, specifically https://help.github.com/articles/creating-project-pages-manually/

I fiddled around in WebStorm yesterday trying to figure out how to set up my own local gh-pages branch, but I failed. It's probably another "so simple Marc can't see it" things. I'll try again today.

commented

Ok, I figured out my gh-pages issues. Here's the result: http://sympmarc.github.io/SPServices/ See the link toward the bottom:

We're in the process of migrating the docs from Codeplex here.

Now I need to understand how to get the .md file to render as HTML.

commented

I saw that...
So you are going to keep the documentation in the gh-pages branch, is that right?
I ask only because that might breakup your development flow... example: You are updating a new utility that also has changes to documentation... it maybe desirable to also update the documentation at the same time and include it all in the same commit... Having the docs in the gh-pages branch makes this a 2 step process (with several sub-steps)... it also makes it harder for your contributors to submit changes. What I was alluding to with my prior response was for you to keep the docs with the actual source code for SPServices... I think that might make your flow more streamlined. It will also allow you to properly version the documentation along with the code - you would not want your "in development" stuff showing up on the real docs, if the code was not yet ready.

Converting documentation to a web site

So now your question must be: How do get those into the gh-pages branch so that they show up on the nice site that GitHub allows you to have per project.

At a high level, your gh-pages branch will host a web application - that is separate from your SPServices source... My view is: That web application will have a build process (Grunt) that will create the documentation from Markdown. The build script would have the following tasks to perform:

  1. Download the desired version (tag) of SPServices source code, which include the Docs. I say version/tag, but it could also be the latest from the MASTER branch.
  2. Convert all the .md files to .html files
  3. Commit the newly created .html files to the gh-pages branch - Which essentially makes them go live in the web site.

Does this make sense?

It sounds like a allot of work, but remember: once it is setup, its hands off... you just run a build target/task and it does the rest.

Documentation organization

How is this for organization and folder structure:

docs\
        index.md
        core\
                index.md
                api\
                      index.md
                      ...
                      Lists.md
                      Lists.imageExample.jpg
                      Lists.GetListItems.md
                      ...
        utils\
                index.md
                SPGetCurrentSite.md
                ...
        value-added\
                            index.md
                            SPCascadeDropdowns.md
                            SPCascadeDropdowns.image1.jpg
                            SPCascadeDropdowns.image2.jpg
                            ...

This is only a rough suggestions based on what it looks like in codeplex today... Obviously, if any one directory gets too big, you should probably break up into further sub-folders...

Notice how each folder has an index.md - that might allow you to then build a "site" map (again, a script) that points at each index file... that index should probably provide a Table of Contents for all the docs found under that folder (again: this can be scripted and auto-generated).

Also notice how I suggest images in the docs be stored - along with the respective help file (same folder) and named very similar to the doc. file. This helps when creating docs because you don't have to mess around with the image paths.

Ok.. that's all the time I have now... I will not forget about this... :) will come back when I get another chance. Keep up the good work.

/Paul

commented

As usual, your thinking on this is solid, Paul. I like the build idea, and I think it also gives us options to do things like creating a printable set of docs (people ask for this from time to time). Keeping the docs in the same repo as the src means we can version them by branching for new releases, too, right?

The one thing I seem to be getting stuck on is the local authoring process for md files with images. I'll figure it out. I can write html in my sleep, but md is a slightly different flavor I'll have to get used to. This online converter seems to work pretty well: https://domchristie.github.io/to-markdown/, though I've only done one doc file so far.

I'll start the docs directory in the master branch and we can go from there. The build process can come later.

commented

Re: Branching
Correct... it keeps it all together which means your docs will always match what's in your code... it also gives you the ability to host multiple versions, if wanted it... Very flexible.

Here is an example of how you can include an image in Markdown:

Look... this is me: 
![Paul Tavares](https://avatars2.githubusercontent.com/u/1473204?v=3&s=96 "paul tavares")

Will output this:

Look... this is me:
Paul Tavares

Also, remember: you can include html in Markdown files... that's not a problem... Parsers will recognize it and pass it through...

So for the structure I suggest above, if you are referencing an image in the same folder as the file itself, it would look like this:

Setting up the SPCascadeDropdown looks like this:

![Example 1](./SPCascadeDropdowns.image1.jpg "SPCascadeDropdowns sample 1")

FYI... I usually use this to convert manually: http://daringfireball.net/projects/markdown/dingus
Its the original markdown and thus does not support "Github flavored Markdown"

commented

I like where this discussion is going! That directory structure looks sound, and the naming convention and location for images is smart. What about if another image gets added in between existing images in the future? If they are named with a number in sequential order, does that matter? Just increment the number for each new image added and not worry about the order they are used in the documentation?

Once some of these decisions and conventions are settled, I think the repo could benefit from a CONTRIBUTING.md file.

@purtuga I use that same markdown resource quite a bit 👍

@sympmarc you're supposed to write in markdown because it's faster/easier than writing HTML, not the other way around! ;)

commented

Nice to hear from you, @joshmcrty! Everything's harder until it isn't. ;+)

What about images that are shared across docs? For instance, the little certification badges are sprinkled throughout. On one level that's an argument for a /docs/images directory, but maybe only for the common images?

commented

Hey @joshmcrty , nice to hear from you..
I don't think the name of the image name matters... I suggested a name that starts with the name of the associated documentation file just to get an idea of what the image is about when you look at a folder structure, but even that could deviate. Inserting images in between should not be an issue, since they are not part of any automated process that expects them to be in a certain order/format... as long as the Markdown image tag points to a valid file, that should be all that is needed.

@sympmarc I agree... any common type of image, that is used in multiple pages, should go into a high-level images folder.

commented

Good convo. Thanks for reminding me of it, @purtuga. I'm going to rough the docs directory out a bit better so that we can move forward...

commented

I'm getting this markdown thing figured out.
This will come in handy - note the checkbox up top for GitHub Flavored Markdown
http://domchristie.github.io/to-markdown/

The main question I have is whether it would be faster to:

  • Copy over all the files as-is and then convert
  • Copy them one at a time

The former is appealing because we can do wide-scale replacements. On the other hand, the files would be ugly longer.

p.s. I can grab the HTML as an admin on Codeplex from each page sort of easily, but I wonder if I could suck out all the pages using some sort of tool.

commented

I propose that they be moved over one at at time... and into the new format. To copy them over as is, I don't think busy you much - other than maybe the hassle of having to go over to CodePlex to update the docs with the first release, so why go through the hassle of copying the HTML of each into a file here..

Re: Generating documentation system from .md
Over the holiday break, I played with metalsmith.io. I installed it on SPWidgets, did a little setup and ran it against the documentation folder... Surprisingly: it worked... Output all the files in markdown. That's where I stopped... my next step is to figure out how to pluing in a page template and output entire documentation system. The tool itself seems very light and simple to use.

The output can then be packaged into the zip file (documentation delivered with the distro), a nd also copied over to the gh-pages and hosted on your SPServices web space.

commented

I spent some time yesterday moving files over. It's not that bad, but it does take some manual work.

A workflow that seems useful:

  • Open the doc page on Codeplex in edit mode (only an admin can do this). Alternatively, someone could copy the HTML from View Source, but they would have to peel out the content from the chrome.
  • Copy out the HTML source code
  • Pass the HTML through domchristie/to-markdown -> This gives me a decent first cut at the markdown
  • Create the new GitHub-based .md file and paste in the markdown
  • Manually edit the images, links, formatting, etc. A markdown cheat sheet is imperative for me!

I've done some of the hardest pages, I think. The pages that show individual operations or functions have less formatting to them than the top few layers.

I've been putting the images into /docs/img. There aren't a huge number of them. Maybe later I can clean up the structure to put the images into separate locations like @purtuga suggested.

metalsmith.io sounds great. As usual, Paul, you are paving the way...

commented

An update on this...

I've got most of the doc pages copied over and converted to markdown. I've been working on getting a build steps running with gulp that:

  • docs - Converts the .md files to .html
  • deploydocs - deploys the html tree to gh-pages

Right now I'm fighting with the YAML headers a bit. For some reason the gulp-markdown-docs parser isn't understanding where the end of the YAML is in .md files I edit manually. I'm not sure what's going on.

This whole thing is an excellent learning experience!

commented

Hi Marc...
Got some free time to help with docs, but not sure where to start... Maybe you can share how to get the documentation up and running... Like is there any local server to start? or how to generate the HTML version of the .md file? Or are you using something else?

/Paul.

commented

Paul:

Frankly, I'm not sure I have the best doc scheme in place. As you can see from my comments above, I don't have a good process in place yet. The docs that are in place here have YAML metadata at the top, but I'm having a devil of a time trying to figure out why gulp-markdown-docs isn't seeing the end of the YAML.

As you know, the goal is to have docs in markdown and then publish to gh-pages. I haven't been spending enough steady time on figuring all this out, as you can see from the gaps in dates. I'd love to get this new release out there and get people contributing here.

Any suggestions on how to make a list of docs and divvy them out? The options I've though of:

  • Add one issue per doc. This seems like overkill.
  • Add one issue with a list of all the missing docs I can identify. Each person who wants to work on one could add a comment with the remaining docs.
  • ???

M.

commented

Lets use this open task to track the list of items that need to be done... Github style Markdown allows for "todo lists"... I just edited the first post above by (you) to include two TODO items... You can add as many as you like and include perhaps who is working on it - case others pitch in... we don't want multiples working the same item (yes: I do envision others will come to help :))

So it sounds like the machinery to get a local server running to view/display the docs is still not there (at least now I know I'm not going crazy, because I did search for it :))... Once you get the markdown generation to html working, that would be the next step... Deploying to github-page is good, but not during development... I look at that step as the last "publish" button.

I have to run... I'll revisit this again...

commented

Thanks, Paul! As always, you're at least a few steps ahead of me...

There's a task in the Gulpfile called "docs" that is the start of it. That SHOULD take the MD files and generate HTML files with a TOC. I can't seem to get past the parser with the YAML thing.

commented

I'll try to look at if I get chance.  Gulp is new to me, but it's all JavaScript :-)

Paul
-- sent from mobile

commented

I messed around with the gulp-markdown-docs plugin and could not get it going either.. I did not spend allot of time, but at first glance, it appears all the issues are being caused by the YAML parser... i looked at the site for the plugin and not much activity there, so could not really see if others have been have hit this problem.
I took two of the files generating errors and used an online markdown editor and it was able to render the file... Maybe its time to look for a different plugin, if you have been spending allot of time on it..

/Paul.

commented

Yeah, I think I've reached the point of diminishing returns. Do you recommend something else? I looked at Metalsmith briefly. I know a lot of others use Jekyl.

commented

Yeah, at some point you need to cut the cord :) YAML is very temperamental... one space is off and the entire thing falls apart.
Jekyl is certainly a great tool and many use it to generate full websites... I think it mentioned this before: it require Ruby I think, so that's another level of complexity you'll have to deal with. Metalsmith (to me) sounds promising, but I have not played with it enough...

I'll see if I have some time later tonight to play with this again...

commented

I posted a question to the creator of the gulp-markdown-docs plugin - sojournerc/gulp-markdown-docs#4

I just don't understand why it is failing... Kind of frustrating... Its should not be this hard. :)

commented

I'm thinking that's the wrong plugin to go with; it's just not working.

I think I'll try this one: https://github.com/sindresorhus/gulp-markdown

commented

Seemed to work on the first try. I've pushed the change to master. See what you think, @purtuga.

commented

Yeah... Based on the response from that other addin - I still don't get it and thus why spend the effort.

I'll try the new one as soon as I can...

commented

@sympmarc @purtuga I've been working on a docs build using metalsmith this morning. I think this is going to be the way to go. I plan on having separate commits for each "phase" of the solution i.e. first commit updates package.json and Gulpfile.js to get it working, second commit adds basic templating, third commit adds auto-generated nav, etc. That way everyone can see how it was implemented and we can roll it back if we want to do something different. Rather than one commit with a whole bunch of changes all at once :)

@sympmarc I see you already committed HTML files output by metalsmith, but package.json doesn't have metalsmith listed in the dependencies. Before I go too far with this, do you have a solution built already that you haven't pushed to GitHub?

commented

@joshmcrty Sorry for the lack of response. I was at Ignite last week and just red flagged a lot of things to get back to them.

I didn't have anything that was worth keeping. I had tried to build some workflow about generating the docs, but never really got there. Anything that you are doing is DEFINITELY better!

commented

Hi Josh. Great to see more contributions. I been lacking in contributiong here... :(I'll be very interested in learning how you use metalsmith. 

Paul
-- sent from mobile

commented

@sympmarc I figured it would be a slow week here with Ignite happening :)

I made some good progress over the weekend:

I found a few wiki pages from CodePlex that weren't in the checklist above so I added them. The one I wasn't sure about was the articles and press page, so I didn't migrate that one yet but I will if you want to keep it, just let me know.

I merged a bunch of commits that clean up the docs .md files, add remaining pages, and update the Gulp docs task to use metalsmith to build HTML pages in the dist/docs folder. @purtuga take a look at those commits or just the Gulpfile.js as-is now and you'll see how I approached using metalsmith; any suggestions welcome! I really like it so far.

I've been working in a fork of SPServices but honestly it was a bit cumbersome and unnecessary since I've got commit access to the main repo. So after those latest commits were merged I created a new branch here called enhance-docs and made further updates to the Gulp deploydocs task so it takes the entire /dist folder, zips it up, then puts the zip file and all the HTML docs into the gh-pages branch. As of now http://sympmarc.github.io/SPServices is the result of that task. I want to keep working in that branch to make the documentation better before merging back into master (e.g. add the SPServices logo, better navigation, etc.).

commented

Just checked out the Pages site: Wow... That is looking great... Awesome
job Josh.

Paul T

commented

In the enhance-docs branch I've made some updates to gulp that should close out this issue.

  1. The docs task now uses metalsmith-watch to watch for changes to the source *.md files and rebuilds the HTML files. We probably won't need to run this task manually because:
  2. I created a servedocs task that runs the docs task and starts a local server with browser-sync to view the docs. It will reload the browser when any dist/docs files change, so we can instantly see how changes to *.md files will look.

Caveats

The docs task does a one-time clean of the dist/docs directory right before it runs to clean out any files that aren't being used anymore. This means that if you delete a .md file while it is watching, it won't remove the output .html file. You'll have to manually run gulp clean:docs to clean out any orphan files (or re-run the docs task).

commented

@joshmcrty You're awesome!